I recently moved on to a new company, but my prior company had a pretty large scale Elasticache Redis deployment in production (over 50 large clusters in us-east-1), and were in the middle of a complete migration to Valkey due cost savings, improved performance, and reduction in memory usage.
We've already completed migrating several large production clusters and I can confidently say that the migration had been pretty smooth and seamless.
Valkey is certainly production ready (at least on AWS it is). The team is looking forward to expedite and complete the migration
Congrats! Love seeing an alternative to Elastic/Lucene, esp. in Rust! Just wondering, for those using Elasticsearch for log storage/querying, could Meilisearch potentially fulfill this use-case also? And are you aiming to be a more "performant" and efficient alternative to Elasticsearch?
Thanks kevinsf90 :D
Meilisearch is really focus towards `end-user search` or `customer-facing search` so, there is few chances we go in that direction.
But we are really aiming to be the easier, simpler and thus more performant alternative to Elasticsearch. Since Meilisearch won't cover every uses cases that Elasticsearch covers, it is much more performant and intuitive on this subset.
If you want to build complex queries, with terabytes of data & aggregation queries you should definitely use Elasticsearch. If you want to build the best search experience for your end-user that will be the most relevant and answers in a few milliseconds, then Meilisearch is the obvious choice here :)
Meilisearch is rather the free and open alternative to Algolia, as it focuses on solving zero-config and easy to deploy autocomplete search, as opposed to analytics at scale.
We're thinking about a few different models, but haven't settled on one yet. What kind of model would you be interested in? Metered or other usage-based, per-seat or per-user, or something totally different?
For a large codebase, these upgrades will be a pain, especially on ruby/rails. To scale in the long run, it'd probably be wise to modularize & split the codebase into microservices, and at the same time, port to, say, a scala or java based framework (like Play).
I'd be interested in hearing a bit more about how GitHub structure their app. From the sounds of it, they have one big monolithic app. Running the tests can't be pretty on that...
That's awesome! We were using parallel_tests, but we just bought a second test server. I was looking into Kochiku from Square, but that gem looks perfect. Thanks!
Workday Inc. - Data Scientist (full time) - San Francisco
We are looking for some smart data scientists to join my team at Workday.
In particular, we're seeking a curious, out of the box thinker and interdisciplinary data scientist to work as an applied machine learner and algorithm designer. You will bring your expertise to efficiently extract patterns and insights from millions of rows of transactional data to help optimize our workflow and come up with data-driven products using machine learning to help customers in optimizing their business.
About the Team
As part of the Data Science team, you will partner directly with the core decision makers at Workday to help optimize customer experience and help answer business questions. You will work with some of the smartest data scientists to analyze user transaction data and build machine learning models to optimize search, recommendation and personalization across the suite of Workday products including HR, Financials, Payroll and others. You will work on delivering actionable insights and predictive models to help answer business questions around organizational and employee performance.
Responsibilities
• Proficient in translating unstructured business problems into an abstract mathematical framework.
• Excited to learn and apply new methodologies in the intersection of applied machine learning, computer science and statistics and make approximations where needed to build scalable algorithms.
• Excellent interpersonal and communication skills and ability to convey concise and actionable story through data to different parts of the company.
Requirements
• MS/PhD degree in Statistics, Computer Science, Operations Research or related field.
• 2+ years of work experience with proven track record of data science and/or algorithmic development.
• Proficiency in at least one high level programming language like Java, Scala, Python or C++.
• Experience in large scale data analysis in Pig, Hive or Spark is a plus.
• Proficiency in atleast one statistical modeling tools from among R, Matlab or Weka is a plus.
• Experience in predictive analytics and machine learning algorithms especially for supervised (e.g. SVM, Logistic Regression, Boosting) and unsupervised (e.g. k-means, LDA, EM) methods.
We've already completed migrating several large production clusters and I can confidently say that the migration had been pretty smooth and seamless.
Valkey is certainly production ready (at least on AWS it is). The team is looking forward to expedite and complete the migration