Why would I use MemSQL as opposed to relational database on SSD? RDMS on SSD gets me a similar speed boost as opposed to running on a completely new database.
Thanks for the responses. I have a few follow ups I'd love to get your perspective on.
+ DRAM is actually much higher on the memory pyramid than SSDs, so MemSQL handily beats MySQL on SSD any day
DRAM is an order of magnitude slower than SSD's. It's also an order of magnitude more expensive, however. You can get a TB of SSD for about $2k, which means 50TB of SSD for $100k, or the price of a single engineer for a year. Meanwhile, a Quad High Mem EC2 instance (64GB of Memory is around $1,700 a month), or $20k a year.
Thoughts?
+ Putting MySQL on SSD doesn't change the fact that MySQL isn't optimized for SSDs or DRAM; so you're still using b-trees, latching and locking, etc.
True, but there are still a number of optimizations that can be made.
Issues like locking become a problem when IO is slow, and transactions have to wait for locks to clear. When locks are 3 orders of magnitude faster on SSD's vs spinny disk, those issues tend to fade away.
+ SSDs have poor write endurance, meaning it has low number of write cycles
Agreed, SSD's have worse write endurance than RAM. You should be able to architect around that fairly easily, however.
That being said, I spoke with Blekko's CTO, and out of the 700 SSD's they purchased 2 years ago to serve their index off of, they've had 2 drive failures. Their index is write only.
+ You would still have to shard your database yourself; while MemSQL helps manage that process
Not having to shard is a Good Thing(TM). But, if you can throw 10TB of SSD storage in a server, sharding should be less of an issue unless you're dealing with Google(n) sized data problems, no? I suspect that 10 TB of DRAM MemSQL would be cost prohibitive.
+ DRAM is available on the cloud today while it will be some time before SSDs are offered on most public clouds.
Absolutely correct, but SSD's are rapidly replacing spinning disk in Enterprise data centers. I suspect that the first VPS service that offers SSD storage is going to rapidly kill the competition because of the insane performance differential. How long am I going to be willing to pay an extra $10 a month for 1 GB of extra RAM in my VPS when I can purchase an 80 Gig SSD for $120?
+++++++++++++++++++++
I'm not trying to troll or be difficult. I'm honestly looking for answers to the in memory / NoSQL / MemSQL vs SSD debate. I've looked at the issue from a number of different angles, but I keep coming down on the SSD side. I figured you would have some of the best, most thought out rebuttals to the issues I've been thinking about.
But, everybody seems to be using Cloud VPS's because they seem very wary of putting some hardware together and throwing it in a rack. I suspect that because I've been rolling my own PC builds for more than a decade, I'm probably much more excited about throwing 20 SSD's in a 2 x 8 core server to see what it would do than someone who has purchased Mac laptops for most of their career. (not flaming, just thinking out loud).
I have a theory that a side effect of Apple owning the startup developer market is that developers think of servers as something residing in a cloud to be rented rather than built from parts. And, because of that, things like NoSQL and non-relational databases have gotten a ton of buzz rather than throwing a _little bit of money at hardware to solve the problem.
Typical relational databases are designed from the ground up with the assumption that their disks behave like a traditional disk (hi-seek time). Whatever speed boost you're getting from using an SSD is far under the potential.
Why would I use MemSQL as opposed to relational database on SSD? RDMS on SSD gets me a similar speed boost as opposed to running on a completely new database.