Abhishek mamidi - DS

Can we explain the output of complex tree models? We use different algorithms to improve the performance of the model. If you input a new test datapoint into the model, it will produce an output. Did you ever explore which features are causing to produce the output? We can extract the overall feature importance from the model, but can we get which features are responsible for the output? If we use a decision tree, we can at least explain the output by plotting the tree structure. But, it’s not easy to explain the output for advanced tree-based algorithms like XGBoost, LightGBM, CatBoost or other scikit-learn models. To explain the output for the above algorithms, researches have come up with an approach called SHAP. SHAP (SHapley Additive exPlanations) is a unified approach to explain the output of any machine learning model. SHAP connects game theory with local explanations, uniting several previous methods and representing the only possible consistent and locally accurate ad

If you haven't read the Part 1 and Part 2 of my Data Science journey, I recommend you to read it before you go through this article. Link to the Part 1 and Part 2 . In the sixth semester, I worked on multiple Deep Learning projects. Neural Artistic Style Transfer was one of my favourite projects I have worked on. I have used TensorFlow in this project and this was my fist Computer Vision project. I enjoyed doing it. I felt that I had a decent number of projects on my profile. So, I started applying to Data Science internships through LinkedIn. I requested my connections on LinkedIn to refer me in their companies. I received calls from a few companies. I gave interviews and ended up with two offers, one from a startup and another from a well established company. After doing research and reviews on glassdoor, I accepted the Data Science Summer Internship offer from Monsanto . It was a unique experience giving interviews from various companies. I am fortunate to get an inte

If you haven't read the Part 1 of my Data Science journey, I recommend you to read it before you go through this article. Link to the Part 1 . In the early days of my third year (5th semester), I decided to have a career in Data Science, after taking feedback from my relatives, professors and seniors. At that time, I heard only three words on the internet i.e, Artificial Intelligence, Big Data and Cloud computing. The Machine Learning course by Andrew Ng was awesome and showed me a path to explore a different domain. My performance was good in the first four semesters, so I got an opportunity to work under a professor for two years in a specific domain. I didn’t want to lose this opportunity. There were two main streams in my college, Natural Language Processing and Computer Vision. There was a huge competition for Computer Vision in my college as there were many professors working on Computer Vision problems. I never explored both domains. Finally, text processing and