Do you love working with technologies like ElasticSearch or Solr at scale? DeepCrawl is a sophisticated web site crawling and auditing tool, and we’d love for you to join our team to help build our sophisticated reporting engine.

Our many customers depend on DeepCrawl to improve user experience, manage change, and improve search engine performance. At DeepCrawl, you’ll have great opportunities to learn about large-scale web crawling, and how to build a complex Ruby application. You can also use 10% of your time to learn or build anything you want.

Our team is built foremost on constant learning, supporting each other, and loving working together. A great team environment supports us in building great technology. We’re always exploring new ideas and tech. Our stack now includes Ruby, PostgreSQL, MongoDB, a sophisticated API using Sinatra, and a front end in AngularJS.

Responsibilities will include:

  • Collaborate with platform engineers to build a sophisticated reporting engine using
  • ElasticSearch or Solr.
  • Work with the product team to find new ways to deliver powerful features for users
  • Develop benchmarks to determine the optimal way to structure and query data
  • Learn new technologies and practices, and provide support for other developers

Skills & Requirements


  • A positive character and an ability to collaborate well with others
  • Solid professional development experience
  • A deep knowledge of ElasticSearch or Solr
  • Experience working with data at scale
  • Solid experience with modern testing practice
  • Excellent spoken and written English


  • Good knowledge of Ruby
  • Experience with Java
  • Knowledge of PostgreSQL
  • Skilled at optimising relational databases and queries


  • Flexible Working
  • 10% time on your own projects or learning
  • Share options
  • Free fresh fruit delivery

About DeepCrawl

DeepCrawl is a sophisticated crawling service, used by the world’s largest brands and SEO agencies to find and monitor many problems with their sites — ultimately making them easier to find and use. Our team emphasizes friendliness, learning, and collaboration. We’ve got a diverse range of backgrounds, and our interests include classic & independent film, life drawing, crafts & interior decoration, photography, motorcycle racing, hiking, and good Japanese whiskey.

We believe that by providing powerful tools to improve web sites, we make the web a better place for everyone. As an established but still fast-growing startup, we’ve got big ambitions to continue our growth and expansion into new markets, and be the tool of choice for SEO, digital marketing and beyond.

Tagged as: big data, data engineer, elasticsearch, solr

Cntrl/Cmd + Click to attach multiple files.
I confirm I am based in the UK and DO NOT require sponsorship.