the art of equilibrium; embracing structural serendipity
Bi-Monthly Update: May & June 2023
hey there 👋
hey, I’m Dev, and if you’re new to my bi-monthly newsletter, welcome! My bi-monthly newsletter is where I recap what’s been going on in my life and share some thoughts and reflections from the last couple months in. Allow me to introduce myself, I’m currently a Machine Learning researcher, working in Medical Imaging x AI under Dr. Tyrrell; looking to integrate Artificial Intelligence into a clinical setting to improve the diagnosis process. Over the past couple of months, I have had the opportunity to close a few significant chapters of my life while simultaneously embarking on some exciting new adventures.
I started an Internship at a Machine Learning company in the states, began the first phase of implementation on my research project, started a new ML-based substack publication, became the director of Project X at UofT AI, and much more…!
These last 2 months have been extremely busy, but by far the most fun I’ve had working in a while, but now it’s time to sit down and reflect on the progress I’ve made! As I reflect and share some key insights into my life, I hope this newsletter adds value to your life as well. Now let’s get into the thick of this newsletter!
interning @ Interactions 👨🏽💻
This summer, I’m going to be interning at Interactions as a Machine Learning Engineer intern. To give some background, Interactions is a leading company specializing in the development and implementation of advanced conversational AI solutions. With a focus on transforming customer experiences, Interactions LLC leverages cutting-edge technology to deliver natural language understanding and interactive communication solutions for businesses across various industries. Throughout the course of my internship, I’ve had the chance to work with some cutting edge conversational AI technologies and I’m super pumped to be working with the team this summer.
I’m about halfway through my internship and I’ve been working on some really cool things which I’d love to share. One of the main deliverables that I’m working on is working with NVIDIA’s Audio2Face interface to create a captivating live avatar capable of engaging in interactive conversations. This innovative avatar serves a distinct purpose: to provide exceptional customer support, particularly for installation assistance. By seamlessly integrating cutting-edge technology with empathetic virtual assistance, we aim to enhance customer experiences and streamline the process of troubleshooting installations. Here’s a quick demo of the NVIDIA Audio2Face scripts that I’ve been working with:
I’ve had the chance to work with technologies that I’ve never worked with before, but the learning process has definitely been worth it. Working alongside a talented team at Interactions has further enriched my internship journey. Collaborating with industry experts and passionate individuals has provided a nurturing environment for personal growth and professional development. The vibrant atmosphere fosters innovation and encourages exploration of new frontiers in conversational AI. Each day brings new insights and challenges, igniting my curiosity and driving me to push the boundaries of what is possible. As I enter the final 2 months of my internship, I’m super pumped to finish these projects up and walk away with something I’m proud of and can’t wait to share the final product with everyone!!
becoming director of Project X at UofT AI 🤖
Another major update is that I’m the new director of Project X at UofT AI. UofT AI is UofT’s biggest AI/ML group that is working towards getting students involved in the growing field of Artificial Intelligence. UofT AI has 3 main components, an undergraduate research competition (Project X), a simulated reality conference (UofT AI conference), and an introductory machine learning educational program (Learn AI). I’m going to be taking over the Project X team as the director and I’m super pumped to make this one of the biggest undergraduate research competitions! For some context, every year UofT AI hosts this undergraduate research competition and universities from all over North America come and participate. Over the span of the 5 months, the teams work and submit a final paper, which is eventually assessed by a panel of academic & industry experts! These next 2 months will be super hectic with getting sponsorships for Project X, reaching out to universities, recruiting a team, and finalizing a theme. As someone who’s been in the Machine Learning field for a few years now, I’m super pumped to extend this opportunity to both experienced and inexperienced people in this field.
While I have always been the kind of person who enjoys rolling up their sleeves and delving into hands-on work, stepping into the role of a director is an entirely new experience for me. The allure of getting my hands dirty, tackling challenges head-on, and experiencing the sheer joy of discovering insights has been my driving force. However, this role is a breath of fresh air — transitioning from a realm of individual exploration into a world revolving around orchestration and guidance. I’m super excited to embrace this fresh perspective; one that is encompassed around strategic foresight and collaboration. This role feels akin to an intrepid painter staring at a blank canvas. As the director, I have a lot of freedom to shape this competition in alignment with my vision, similar to the artist who holds the brush and decides the strokes that will bring life to their masterpiece. My vision for Project X extends beyond a mere competition. I aspire to create a platform that not only showcases the exceptional talent and ingenuity of young researchers but also fosters meaningful connections and collaborations within the AI community. Make sure to keep a look out for updates in the coming months, I will be sharing updates and insights on the progress of Project X! If you’re interested in participating or getting involves, check out the UofT AI website below for updates 👇
research update 🔬
For those of who are new here, let me give a quick update! For the last year or so, I’ve been a part of a Machine Learning x Medical Imaging lab run by Dr. Pascal Tyrrell at the University of Toronto. Over the course of the last 4-5 months, I’ve been working on a research project that aims to create an ultrasound model to downstream tasks for knee ultrasounds. The overarching problem is that ultrasounds are very diverse and heterogenous and it’s often difficult to diagnose accurately. Moreover, ultrasound imaging is highly dependent on the expertise of the technician performing the exam, which can lead to variability in results.
My approach to this problem was to use self-supervised learning, more specifically contrastive learning. My approach is a multi-model approach, I’m combining the usage of multi-scale CNNs and Contrastive Learning. To put it in simple terms, multiscale CNNs take an input of the same image at different scales. The entire idea of multiscale CNNs is to extend the capabilities of traditional CNNs by incorporating multiple scales of information. By leveraging a range of receptive field sizes, multiscale CNNs are designed to capture both local and global features simultaneously. This enables them to better understand the context and relationships within an image or data.
I’m going to be using this, combined with contrastive learning to tackle this problem around ultrasounds. I spent most of May theorizing these models, trying to figure out a viable approach to ensure that the ultrasound inputs can be effectively utilized by the contrastive learning model. Some quick background for those who don’t know what contrastive learning is; the main idea of Contrastive Learning is that it trains the model to differentiate between similar and dissimilar pairs of images by mapping them to a shared latent space.
Recently, I’ve begun to implement this model and have been training it on a sample dataset → so far, I’ve made some promising progress and I’ve gotten the multiscale model to work after lots of testing and experimenting. The feeling of getting a small portion of the code working is definitely one of the most underrated feelings in the world 😅. But going forward, I’m going to work on getting the contrastive learning model working → the next step after this would be to test on the curated dataset I have and get working on the final research paper!
machine learning substack publication ✍🏼
Bouncing off the topic of the last section, over the course of the last 2 months, I’ve started a new Machine Learning publication. I call it AI bits & bytes and the main purpose of it is to cover new papers in the machine learning space, but it also serves as a way to keep people updated about my research :) I’ve been writing about papers that have not only deepened my understanding but also shaped my research approach. It is a platform through which I aim to share my insights and learnings, offering a public forum to disseminate knowledge and foster meaningful discussions. Currently, I’ve made 2 posts on the publication, covering the 2 topics I mentioned in the previous section, multiscale CNNs & contrastive learning. I plan on posting 1-2 times a month and if you’re into the machine learning space, or just want to learn more about ML, drop your email down below and consider subscribing :)
building in AI 🔨
For those who truly know me inside and out, they understand just how crucial it is for me to maintain a sense of balance in every aspect of my life. Among the myriad of passions that ignite my curiosity, there is one flame that never seems to wane—the relentless pursuit of knowledge and continuous learning. Learning has always been my driving force, propelling me forward on my journey; earlier in the year, I spent a lot of time building machine learning models and understanding the intuition + math behind why these models work the way they work. One thing that I realized was that I made these models, but I wasn’t producing some sort of final product that I can use in my day to day life. And so, I spent a lot of time learning how to make full-stack machine learning projects and I created 2 that I’m able to use in my day to day life. I created a photo restoration web app and a back posture feedback web app.
Now that I reflect on it, these projects have become a testament to my personal growth and commitment to applying my knowledge in a way that directly impacts my life. They have allowed me to merge my passion for machine learning with my everyday activities, bringing a newfound sense of fulfillment and purpose.
photo restoration web app.
The photo restoration web app is actually one of the more helpful ones I made, especially when I find blurry pictures and I need a clearer picture. I made the entire project using python, html, flask and machine learning models built with pytorch. I made the backend with python; I used flask for the web app and wrote some functions to read the input file and call the API of the ML model that is used to restore the quality of the pictures. This was by far one of the most fun I’ve had building, I learnt how to use flask for the first time, but debugging the code was definitely a learning experience 😅. If you want to check out the code for it, I’ve linked the code in the button below, in the README.md file, I’ve linked a live demo of the video, be sure to check it out!
sitting posture feedback web-app.
My sitting posture is generally not the best and I was too lazy to actually fix it (as you can tell from the picture), so I built a web app that gives me feedback on my back posture and tells me how to fix it. Building this web app was really challenging, I kept running into problems about the coordinates of my shoulders and my chest. But I wanted to flex my bandwidth a little more and instead of building the web app with flask and python, I built the backend with JavaScript and using the p5.js libraries. After almost a weeks worth of testing, I got the web app working and I hosted it through GitHub. I plan on adding more features to it, for ex. having a separate setting to judge your standing posture as well! If you have any feedback, feel free to reach out! Click the button below to check out the web app, it’s hosted and free for anyone to use :)
attending collision 2023 💥
Last week I got the chance to attend North America’s biggest growing tech conference and it was a surreal experience. From networking with industry experts to watching the Godfather of AI give a talk on the future of AI, it was a great experience full of lots of valuable insights. Attending conferences generally leave an indelible mark on my personal growth — being surrounded by like-minded people is both a humbling and empowering experience. The energy and enthusiasm in the air create an atmosphere of possibility and excitement, igniting a renewed sense of purpose and determination within me.
Networking with industry experts provided me with a unique opportunity to connect with individuals who are at the forefront of technological innovation. Engaging in conversations and exchanging ideas with these professionals not only expanded my knowledge but also challenged my perspectives. It reminded me that no matter how far I've come, there is always more to learn and explore. The highlight of the conference for me was watching Geoffrey Hinton take the stage and talk about the future of AI. As someone deeply immersed in the ML space, it was a truly enlightening experience to hear his objective perspective on the trajectory of artificial intelligence. It provided a broader context for my own work, allowing me to see beyond the immediate projects and challenges I had been focused on.
These conversations that came out from the conference aren't average conversations you have with people, and it really helped me realize the depth and richness that can be found in engaging with experts in the field. Each interaction was like discovering a hidden treasure, where insights and ideas flowed freely, creating a dynamic exchange of knowledge. Moreover, these conversations acted as catalysts for personal growth and development. They provided a platform for me to articulate and refine my own ideas, to challenge and be challenged, and to broaden my perspectives. Overall, the conference was a great experience, full of continuous growth and unforgettable connections. Super pumped to be back here for Collision 2024 and connect with more people 💥
structure is key 🔑
Since the last time I wrote, I’ve been on summer “break”, but it’s felt like the complete opposite. Although I've had more freedom, having some sort of structure and routine is crucial for me to stay focused and productive. Without it, I've found myself drifting aimlessly, lacking the drive and motivation that comes with a clear sense of purpose.
Many of the challenges I faced during this time can be traced back to the pervasive distractions that arose following the break for summer. Although these diversions provided momentary satisfaction, they ultimately left me feeling unfulfilled and disconnected from my aspirations. Recognizing the value of my time, I swiftly realized that investing it in purposeful endeavours rather than succumbing to these distractions would lead to a more meaningful & worthwhile journey. Having embraced the previous value of time, I made a conscious decision to establish intentional boundaries and reclaim control over my focus and energy. These changes weren’t drastic, but they had a profound impact on my productivity and overall well-being. I realized that even small adjustments in my day-to-day routine could yield significant results.
During this break, I've also had the opportunity to explore new hobbies, catch up on personal projects, and spend quality time with loved ones. It initially felt liberating to have unstructured time, free from the demands of work or academic responsibilities. However, as the days passed, I realized that without a defined schedule, my days started to blend together and I lost a sense of direction. In a way, it reminded me of a ship at sea without a compass. Without a clear path to follow, the ship meanders aimlessly, unsure of its destination. Similarly, without a structured routine, I found myself lacking a guiding compass, unsure of what tasks to prioritize or how to make the most of my time.
For me, juggling research, an internship, and summer school simultaneously has presented a significant learning curve. In a way, it’s underscored the importance of having a defined structure in order to effectively manage my time and responsibilities. By establishing a well-organized routine, I have been able to navigate the various commitments with greater ease and efficiency. With each day, I’m able to establish a clearer and better routine that aligns with my overall well-being and also brings a sense of fulfillment. It provides the scaffolding that supports my personal growth and enables me to make the most of my time and energy. Rather than feeling constrained by a structured routine, I have discovered a newfound sense of freedom within its boundaries. Embracing structure and routine doesn't mean sacrificing spontaneity or the joy of exploration. Rather, it serves as a guiding framework that brings balance and clarity to my days. It allows me to make intentional choices about how I spend my time, ensuring that I prioritize activities that contribute to my personal growth, well-being, and overall fulfillment.
While only being halfway through summer, I’ve come to appreciate the intricate dance between structure and freedom, discipline and spontaneity. The dichotomy that once seemed conflicting has now merged into a harmonious balance that propels me forward. To put it in better words, structure is not a rigid cage, but more so a vessel that can carry my aspirations and dreams by providing the stability and direction to navigate the unpredictable tides of life.
looking ahead.
If you’ve made it this far, I would like to thank you for taking time to read my newsletter. I hope that my insights and experiences have been valuable to you, and I look forward to sharing more of what I’m up to in the future. With that being said, here’s what I’m going to be working on in the next few months:
Finishing up my 2nd internship — going to hopefully have a working demo by the time I write my next newsletter.
Finishing my proposed model for research — already have a half-working model, but looking to get that last part done.
Keeping up with writing — I’m going to continue to consistently put out articles and pieces of writing.
Getting ProjectX up and running — now that I’m director, super pumped to bring my ideas to life with projectX.
That’s all from me; if you enjoyed reading this newsletter, please consider subscribing and I’ll see you in the next one 😅.