slash n: intelligence @ scale

Let me start this blog with a note of thanks – to you, the engineer! Whether it’s an engineer in San Francisco who open sourced a framework, or an engineer in Beijing who came up with a new technique, or an engineer in Bangalore whose commits increased Hadoop’s performance by 10% – its your work that allows us to focus on solving our problems than being distracted by solving everything under the sun. Flipkart wouldn’t have been here without the support of open source tech community worldwide.

For us, slash n, our flagship technology event, is a celebration of that liberating engineering philosophy of sharing. Building India’s largest e-commerce platform, we’ve learnt a thing or two, created a few pieces of technology, and figured out what techniques/processes work and what don’t. We are committed to share our learnings and to open up our technologies and platforms to the community. slash n is a forum for us to do so and also the forum to learn from the experience of others in the industry, who share the same philosophy about technology.

On Mar 7th, we had the 2nd edition of slash n, and what a day it was! Over 500 engineers participated in the event, out of which half were external invitees. That’s more than double the number of participants we had last year. Considering it was still an invite only event, the interest, enthusiasm and participation shown by the tech community inside and outside of Flipkart was beyond expectations. Unlike last year, when we had most of the talks by Flipkart engineers, this year more than half of the talks were by external speakers from diverse set of organizations. Here are some highlights of the day:

  • The day started with Amod, head of technology at Flipkart, reinforcing the importance of sharing in the technology community, and slash n as our means of doing that. He also committed to open sourcing a few key technology elements at Flipkart over the next 2 quarters, in particular RestBus (messaging system for transactionally consistent service interactions), Aesop (change propagation system) and some of our mobile app technologies.
  • In his keynote, Sachin, founder and CEO of Flipkart, outlined the journey of eCommerce in India, the significant problems that got solved, what challenges lie ahead and how technology can address those challenges in future.
  • This was followed by two talks from very different areas – molecular biology and mobile ad network. Ramesh from Strand Life Sciences talked about how they are using advances in computing to make genetic disease discovery available at sustainable cost to everyone. Rajiv from InMobi talked about their efforts around mining large amount of user data to provide better mobile ads. It was interesting to note that uniquely identifying users across devices, apps & browsers remains the holy grail of personalization on the internet.
  • Some of the most popular talks (based on how many people added them to their schedule using slash n mobile app) included
    • Art of Promising by Yogi & Vikas from Flipkart, which pealed the layers off how Flipkart makes and keeps the availability and SLA promise to customers.
    • Soothsayer @ Flipkart by Ananda and Mohit, which talked about internals of Flipkart’s demand forecasting and inventory planning system.
    • Cataloging talk by Utkarsh and Abhishek from Flipkart, which talked about evolution of our catalog from books to 30+ M items today and how the team addressed the issues around scale, availability and agility along the way.
    • Job Scheduling in Hadoop by Joydeep from Qubole, which provided details on issues around Hadoop job scheduling as well as his experience of building Fair Scheduler and Corona Job Scheduler.
    • Participants loved the newly introduced fire talks – 15-minute quick discussion on a very focused tech topic.
    • Another highlight of the day was a panel discussion on hope, hype and reality of big data analytics, which saw healthy debate among data scientists from diverse organizations like Flipkart, IBM, Xurmo, UIDAI and Mayin, which are trying to use big data analytics to solve problems in different domains.
    • Twitter was abuzz throughout the day with participants tweeting their learnings, questions, discoveries at #slashn.

The atmosphere in the event was quite electric with interesting talks and engaging debates, which often continued between speakers and participants beyond the talk.

IISc Bangalore was the venue of slash n Keynote address by Sachin Bansal

Guest talk by Ramesh Hariharan from Strand Life Sciences Panel discussion on big data analytics

The focus for this year’s event was on ‘Intelligence @ Scale’. The theme encapsulates what we are trying to do from technology perspective at Flipkart and the effort was to share our learnings in this direction and to learn from others. We believe that large scale can become a strategic differentiator if we can use it to make the life of users better continuously. And this can happen when large amount of data generated by the user activities can be used, in real time, to make user experience better via systems that are learning continuously from each user interaction. slash n saw engineers from diverse fields like eCommerce, molecular biology, education, social sciences, mobile ad network, cloud infrastructure, etc. talk about their approaches to build ‘intelligence @ scale’ in their respective domains.

Tech bonding at scale! Full room means good engagement

The day has re-established our core belief that knowledge and technology is meant to be shared and doing so can create virtuous cycle of innovation and progress not only for us but also for the entire ecosystem. I hope everyone who participated had some key takeaways in terms of learning (and a few more connections on your favorite social network) and those who could not, can still watch the recording of all the talks on the event website

We would like slash n to evolve into a more democratic and open platform to share knowledge and possibly to collaborate on building technologies for future. See you all at next year’s event – let’s celebrate the freedom at a grander scale and collaborate more deeply to solve problems that matter.

6 thoughts on “slash n: intelligence @ scale”

  1. Hi Himanshu – this events sounds very exciting! are there any plans to share any of the presentation materials? Also, how are the topics for these events selected for the event? Curious given the wide variety from molecular biology, mobile computing, and internal frameworks that are being open-sourced etc. – a very varied mix which in itself is useful for the attendees!

  2. Hello Flipkart, I always get amazed by your website and how well it is built in terms of response speed and UI design. You guys have done an amazing job out there – Is it based on Magento by any chance?

  3. Thanks Regu,
    Thanks for your quick answer. I did go through the link you mentioned. Its nice to get insight into different technologies that Flipkart is using. Few specific questions though:

    1. What is the core framework you are using for frontend? – Is it Zend since you say its
    based on PHP? – If not Zend then which one?
    2. Whats the persistence layer for MySql? Is it Hibernate?
    3. How are you guys dealing with Heartbleed bug? – How does Flipkart in general respond to these issues?

    Thanks and I await your response

    1. Krishna,

      Answers to your questions:

      1. What is the core framework you are using for frontend? – Is it Zend since you say its
      based on PHP? – If not Zend then which one?
      : We have a home-grown PHP based MVC framework. It provides capabilities like A/B, Metrics collection, Logging and instrumentation and talks to a Reverse Proxy (Phantom) that runs on every web server. We even have a custom protocol that the PHP layer and Phantom understand. This is detailed in the Phantom documentation on github.

      2. Whats the persistence layer for MySql? Is it Hibernate?
      : The ORM layer, if any, is not standardized. Teams use whatever suits them best. We however have a MySQL proxy built on Phantom that is used for testing app behavior by inducing controlled chaos. This proxy is however being used only during development and not in production.

      3. How are you guys dealing with Heartbleed bug? – How does Flipkart in general respond to these issues?
      : We have a dedicated App Security team that regularly tests, monitors our apps and systems for security compromises. Teams treat findings of the security team pretty seriously and get them fixed immediately. Along with this we have a very active Dev and Hacker mailing lists where engineers regularly post such information. This awareness helps teams identify and fix such issues.

Comments are closed.