Thursday, October 13, 2011

MongoDB Grails and Copying Collections

I currently find myself working on a project where Grails and MongoDB are the technology stack... in fact that in part is some of the reason for being interested in the project. I thought I would share 2 aspects of my learning so far:

Grails + Mongo

We spent at least 2 weeks trying to understand which approach we were going to take for our persistent tier. The general concensus was to use the Mongo GORM. We started down that path and ran into a bug and then a limitation on embedding. We got amazing support on the bug followed by the next release of the GORM option which fixed our initial document embedding concerns. We ran into more problems.

We switched to the morphia driver... once again... challenges caused lots of frustration. The challenges included getting other plugins to work with this driver, the fact that the morphia "domain" objects couldn't be in the standard grails domain folder, which further complicated mocking. Frustrations with getting full capabilities of getting spring security to work. It turns out that with spring security there are a number of relational assumptions (along with assumptions of objects being domain classes.. meaning they live in the grails domain package space) which cause problems when you are using NoSQL.

The morphia had another limitation we couldn't live with... we wanted the ability to have different types of objects mapped to the same collection. It appears to me that the Mongo efforts in the Grails space are bolt ons to a rational model. You can get it to work when thinking with a RDBMS mindset. They seem to break down with a NoSQL mindset... this certainly could be attributed to user error (meaning me). I believe we gave it a good try... however the project is under the gun and we didn't have time to "play" with it or wait for responses.

Alas we created our own solution... one of our significant constraints was to be prepared for extremely large scale. This had us duplicating a lot of documents not common in RDBMS solutions... along with the fact we needed to control the number of queries. Ideally we want 1 query for a page... certainly for the main page which is high laden with dynamic content.

Mongo Copying Documents - Collection to Collection

I'm really getting to like Mongo... a lot. One of the downsides is there is limited documentation... including limited forum comments on it. Because of this, I've decided to post interesting solutions to problems as I discover them... time permitting. so for todays tip. I recently needed to duplicate a "record" in mongo speak a document in a collection. There is so much information out there on how not to duplicate records that a google search at this point usually points you in the wrong direction. My situation was such that we have a CMS component to our application, which sets a publish day on content. What I needed was the ability to duplicate a record (so the BSON id would need to change and set a new publish day.

The first thing to recognize for those new to Mongo is that the Mongo console (the command line client to Mongo) works with javascript... so get your javascript fu ready... no.. not your jQuery fu... your plain jane generic javascript fu:)

Here is the solution I came up with:

db.NewsItem.find({newsType: "Blah"}).forEach( function(x){ x._id = new ObjectId(); x.publishDay=283; db.NewsItem.insert(x)} );

So it turns out that you can pass a lambda expression to the forEach method of the collection. The trick is to assign a new ObjectId() to the _id, make any other assignments and then insert into your collection of choice... in this case back into the same collection. This comes in handy as well when you want to backup a collection real quick. (yea, I know you could use the mongodump and mongorestore, for a quick temp solution this just seems more convenient)

db.NewsItem.find({publishDay : 285}).forEach( function(x){ db.NewsItemBackup.insert(x)} );

Happy Coding!


Web Designer said...

nice post it is very helpful thank you

seo services india

Harsh said...

Ken, thanks for the post. I'm new to both Grails and Mongo, coming from a traditional RDBMS model and Java background.

The issue with Grails+MongoDB which I'm getting through is this: I can't embed a collection of custom domain objects within another domain. Collections of regular Java objects such as String can be embedded without a problem, but not a custom domain object :(

If you guys were having the same issue, how did you resolve it?


Ken Sipe said...

@Harsh It is easy to embed custom domain objects in mongo. The answer to your question depends on the tool set you are using. For example if you are using Morphia then check out:

we decided to write our own libraries for managing this. This seems worth of a blog post though. I will follow that up.

Mike said...

Ken,thanks for the post.

I was wondering if it would be possible for you to make a post just about html coding and the basics.

Can you do that?

sathya said...

Thanks for sharing your blog.It is really great blog.i am glad to write comments. Great tips.please Updating your blog with new topics.
Hire magento designer

Microsoft Office said...

Microsoft Outlook 2010 download adapts its performance by displaying either a preview of a message or the full message, depending on the network speed. you can use download Outlook 2010 to create IRM-protected e-mail messages and grant others permission to access and modify your e-mail messages.

cisekta said...

Ken, Thanks for sharing it. It's such a useful resource. Mongo is something which everyone needs to work on.

Before that, people do hire Magento developer for that. But now it's time for Mongo

muhammad ibraheem said...

Investment Plans to make money online, Online Jobs can make money online from home, Just Visit

pvahora said...

I read this carefully really nice to read.. thanks for PSD to Magento

pvahora said...

The blog is absolutely fantastic. Lots of great information and inspiration, both of which we all need. Thanks.
Hire Magento Developer

Ava foll said...

The Mongo database is really complex to understand compared to other available in the industry.PHP Developer India

Vipul Shah said...

I am very thankful to the author to write this fruitful information.It is worth sharing for other users.Thanks once again
register your tuition  class online

Lalmahamode Abdel said...

The information written in the article is descriptive and well written.It is also simple to read and understand.Good Read.
31 day fat loss cure

Vipul Shah said...

Thanks for sharing as it is an excellent post would love to read your future post
Computer courses in vadodara

Lalmahamode Abdel said...

I am very thankful to the author to write this fruitful information.It is worth sharing for other users.Thanks once again
31 day fat loss cure

Safal Shiksha said...

The information written in the article is descriptive and well written.It is also simple to read and understand.Good Read.
register your school online