The ML Playbook Series: A Conversation with Manaj Srivastava

Welcome to the fifth edition of The ML Playbook blog series, where we delve into the insights and experiences of industry experts shaping the landscape of Machine Learning.

Our fifth guest, Manaj Srivastava, is a seasoned leader in machine learning with over a decade of experience, including ten years at Raytheon. Currently, he serves as the Head of AI/ML at Sway AI, where he continues to drive innovation in the field.

Manaj earned a BSc in IT before relocating to the US to pursue an MSc in NLP from Carnegie Mellon University. I had the privilege of discussing various aspects of machine learning with him, especially focusing on leadership and career progression.

What inspired you to pursue a career in machine learning and how has your academic background shaped your career?

Well, it was a bit of a random incident, if you will. It was around 20 years ago, back in college in India and we had this project called the Universal Digital Library. The goal was to gather all kinds of books—mostly Indian ones, in various languages—and digitise them. Along with scanning and digitising, we also had to process these texts in some intelligent way. So, not just OCR, but more complex stuff like semantic analysis and automatic summarisation.

Hearing about these kinds of problems for the first time, and seeing that people were working on such advanced topics, felt almost sci-fi to me. I had no idea about this field at the time, but it sparked my interest. I got into the lab and worked there for about a year, and it was hands-on experience with AI and ML from that era. What got me hooked was the challenge of solving tough problems—problems where you don't know the solution right away and can't just write some code to fix them. It was more about exploring and figuring things out, which I found really engaging.

Then, I was lucky enough to pursue a degree in natural language processing (NLP) from Carnegie Mellon University. It was a specialised master's programme, but it covered all aspects of machine learning as well. Being at CMU was a big boost for me. The environment there—working alongside very smart people, having access to top-notch advisors, and being involved in cross-discipline learning—was invaluable. The hands-on research and the overall learning atmosphere really shaped my journey in AI and machine learning. It opened up numerous career opportunities and set a solid foundation for my future in this field.

20 years is a long time, you must have seen some big changes. What would you say were the most influential? 

Oh, absolutely. The change has been huge. Back when I was in college, machine learning was already a thing, but it didn’t have the momentum it has now. In the early 2000s, companies like Google and Amazon had access to massive data, which boosted machine learning, but it wasn’t as effective or ubiquitous as today.

Around 2012, deep learning and new neural networks sparked a major shift. Models like AlexNet, projects like ImageNet, and the use of word embeddings for language were breakthroughs thanks to big data and computational power.

Before deep learning, we relied heavily on handcrafted features, constantly tweaking them. But deep learning shifted the focus to data and model complexity. Then, Transformer-based models like GPT came along, leading to another major shift. ChatGPT was a game-changer, sparking a revolution where breakthroughs seem to happen every few months.

So yeah, the evolution has been phenomenal. Few could have predicted 20 years ago that deep learning would disrupt the field so completely, and we’re still living through that revolution.

Technical leadership is a recurring theme in your career. How do you balance hands-on technical work with leadership responsibilities, and what strategies do you use to mentor and guide your team effectively?

Balancing hands-on work with leadership is challenging as responsibilities grow. You naturally do less hands-on work, which requires mastering prioritisation and delegation. The 80/20 rule is a good guide—focusing on the 20% of tasks that deliver 80% of the impact. This helps manage overwhelming workloads by prioritising high-impact tasks.

Delegation is key. You have to let go of coding everything yourself, and instead, trust your team to handle tasks. This not only allows you to focus on leadership duties but also helps develop the next level of leaders within your team.

Mentorship ties closely with delegation, as both require mutual trust. You need to earn your team’s trust and be there when they need you, just as they should be reliable when you need them. Building this trust and mutual respect is crucial, especially in high-pressure situations. Caring for your team and being available for them fosters a strong, supportive environment, which is essential for effective leadership and mentorship.

You've worked on a variety of machine learning and engineering problems. Can you discuss a particularly challenging problem you encountered and how you approached solving it?

Different problems present various levels of challenges, often involving not just technical issues but also significant hurdles in making them work effectively within a product and ensuring a good user experience. One particularly challenging project I worked on at Raytheon involved cross-lingual summarisation for an information retrieval system. Unlike typical summarisation tasks, this project focused on creating summaries (or snippets) that would help users decide whether to click on search results, with queries and documents in different languages.

The main challenge was realising that standard metrics for summarisation didn’t fully apply here. The context was very specific, and we needed to prioritise user feedback through A/B testing to refine the system. This taught me the importance of tailoring machine learning models to specific applications and user needs, rather than relying solely on general metrics.

Technically, the project was complex because summarisation was just one part of a larger NLP pipeline that included machine translation, speech recognition, and information retrieval. Constant collaboration with other sub-teams was crucial, as our system had to integrate with theirs seamlessly.

Overall, this project reinforced the importance of understanding the specific problem you're solving, prioritising user feedback, and fostering strong collaboration in large, multidisciplinary teams. It was a blend of technical and collaborative challenges that highlighted the need for a holistic approach to problem-solving in the industry.

Given your extensive experience, what trends do you currently see as the most transformative in the AI/ML industry? How is Sway AI positioning itself to capitalise on these trends?

Right now, a big trend is the surge in interest around large language models like ChatGPT, which are everywhere in the news. Another trend, though less obvious, is that companies, even those that haven’t been into AI much before, are now eager to get on board with AI for various applications, to get on that curve and employ machine learning in their products and so on.

At Sway, we’re focusing on these trends by offering an end-to-end AI solution builder that businesses can use. Our product includes generative AI solutions for text-based tasks—like answering questions or extracting information—and also covers other machine learning needs, such as regression and forecasting, to address a broad range of industry problems.

Looking ahead, what do you think will be the next big breakthrough in AI/ML, and how can companies prepare for it?

That’s a tough question. It’s hard to pinpoint a specific breakthrough because the field is so dynamic. Right now, generative AI has been a major trend, but there’s a sense that the initial excitement might be settling. The future could bring more human-like intelligence or even AGI, but predicting when that will happen is uncertain.

For companies, the key is to keep investing in the ability to evaluate AI and machine learning models effectively. Even if the technology evolves, having a solid infrastructure to assess whether these models meet your business KPIs will help you stay ahead. This means building systems that can continuously evaluate model performance and ensure they align with your business goals, which will be crucial no matter what breakthroughs come next.

For aspiring AI/ML professionals, what skills and experiences do you believe are most critical for success in this field? Are there any specific resources or learning paths you would recommend?

For success in AI/ML, a broad understanding of the end-to-end process is crucial, from data handling to model selection and evaluation. Gaining hands-on experience with building complete machine learning solutions will give you a solid foundation. Once you're comfortable with the basics, specialising in a specific area you're passionate about can greatly enhance your career prospects.

For resources:

1. DeepLearning.AI: Offers comprehensive courses on various aspects of machine learning and deep learning, suitable for beginners and advanced learners alike.
2. Online University Lectures: Many universities, like Stanford and MIT, offer free online courses and lecture series on platforms like YouTube. These can provide valuable insights into both foundational and advanced topics.
3. Coursera and Other Platforms: Paid courses offered on sites like Coursera can be a good investment for structured learning and practical experience.

Overall, a mix of hands-on practice and leveraging high-quality educational resources will set a strong path for success in AI/ML.

Out of interest, would this be something you look for on a CV when hiring for your team? Are there specific things you prioritise, or are they just nice-to-haves?

When evaluating candidates, the most critical factors are practical experience and demonstrated problem-solving skills rather than the specifics of their education or the courses they've taken. Here's what I look for:

1. Relevant Projects: Evidence of side projects or significant assignments that showcase your ability to apply machine learning concepts to real-world problems. This could be through personal projects, internships, or contributions to open-source projects.
2. Practical Experience: For experienced candidates, what you've accomplished in previous roles is more important than where you learned your skills. Success in interviews, where you demonstrate your problem-solving approach and technical expertise, is crucial.
3. Problem-Solving Intuition: A clear understanding of how to tackle machine learning challenges and a proven ability to execute those solutions effectively.

Ultimately, while educational background can be a factor, your practical skills and how well you perform in interviews will make the biggest impact.

Reflecting on your own career, what key pieces of advice would you offer to those just starting out in AI/ML, particularly in balancing technical depth with broader industry knowledge?

For those starting out in AI/ML, it's essential to focus on two things: a strong foundation in fundamental machine learning concepts and hands-on experience. While the field is broad and rapidly evolving, mastering the basics of machine learning and deep learning is critical. Working on projects, even self-driven ones, helps build intuition and practical skills.

You don’t need to worry about being an expert on industry trends from the outset—entry-level roles will primarily expect solid technical skills. As you gain experience, broader industry knowledge and leadership will come naturally. Stay updated by following key conferences or industry leaders, but don’t let this distract you from honing your core skills.

So, would you say a university education is essential for a strong foundation in AI/ML, or can self-study also be effective?

Self-study and non-university learning can absolutely help you build strong expertise in AI/ML. While you don’t necessarily need a university degree, I’d advise caution because, in my experience, there’s still a bias toward candidates with formal education, at least, in terms of getting calls from recruiters and hiring managers.

You can absolutely grasp the fundamentals and add value through a wealth of online resources—many of which are free or affordable as mentioned before. Even paid courses and boot camps typically cost much less than a university degree. If you're disciplined and driven, you can learn effectively and potentially match the education you’d get at a university.

However, a significant advantage of formal education is the access to a community of peers and mentors who can guide you. That said, you can find similar support through external courses, tutoring, or online communities like Stack Overflow and LinkedIn.

Ultimately, while self-learning is a viable path, you’ll need to be proactive in seeking out resources and connections and there’s just this lingering bias for a college or university degree, especially in related fields. Although, as I stated earlier, personally when I am hiring, I give very little weightage, if any at all, to a formal degree but more to what a candidate has done and how much they know.

Outside of your professional work, you’ve been involved in many volunteer activities. How have these experiences influenced your approach to leadership and teamwork in your professional life?

While I didn’t enter my volunteer work with any specific goals in mind, looking back, I realise how much it shaped my approach to leadership and teamwork. For over five years, I volunteered with a nonprofit called Horizons for Homeless Children, working with kids at a shelter. One key lesson I learned was the importance of emotional intelligence—specifically, how to connect with children at their level. I recall an incident where two kids were arguing over kitchen toys, this experience taught me valuable conflict resolution skills, which are crucial in a professional setting, especially when managing teams.

Additionally, I volunteered as a tutor at a high school, where I faced challenges working with teenagers who were struggling academically. This role taught me how to engage with individuals who may not respond well to traditional approaches. Building trust and understanding their perspective was vital in helping them succeed.

These volunteer experiences provided me with insights that I’ve carried into my professional life, particularly in dealing with conflicts and fostering collaboration among team members. Even early in my career, I found these lessons to be applicable when navigating challenging situations with colleagues. In hindsight, the learning from these experiences has been instrumental in shaping my leadership style and approach to teamwork.

If you're seeking top talent in the field of ML or looking for exciting career opportunities, don't hesitate to get in touch with our experienced AI & ML recruitment team. We connect businesses with skilled professionals and provide a platform for individuals to explore promising MLOps jobs. Whether you're looking to hire talent, browse job listings, or even feature in our blog series, we're here to facilitate your journey in the world of MLOps. Reach out to us today to learn more!