What to Do When Everything Changes
The Change
I think it was the MangoNovaBot that did it. Mango and Nova are my nephews. We aren’t the same species, they have four legs, claws and ignore you when you call them, but we’re family nonetheless. And, as I so enjoy things that tickle my brain, the simple robotic arm controlled through bluetooth on an iOS app had to incorporate my sister’s cats in some way. Instead of a claw, I added the most adorable Mango and Nova figurines.
The MangoNovaBot came from a challenge I gave myself after I had learned that ChatGPT could code. This was way back in the era of ChatGPT 3.5. Back then you had to copy and paste the few lines of code it could create at a time. It sucked. According to most people. But as someone who had always been impressed by what you could do with coding and who also ended up being terrible at coding and hating it, I was fascinated. The simple asteroid like games I first asked it to make were super glitchy and didn’t work. I’d ask it to fix something and it would fix it but break two other things. But it would fix it. Despite how terrible it was, I couldn’t help but see the potential. I wondered how far I could push it.
So came the idea for the robotic arm. The challenge was to build a simple robotic arm and control it through bluetooth using an iOS app and to only use AI to do it. I had never actually built a robotic arm and I had definitely never built an iOS app. The robotic arm and the bluetooth iOS app controller were built and working 3 weeks after starting the project. And that was while taking a full Aerospace Engineering course load (although I did neglect some of my studying because this project consumed me in the best possible way). Now to tell you I only used AI to do all this would be a lie. I ended up doing some google searches because ChatGPT just could not teach me about pip and would give me outdated code libraries no matter how confidently it insisted it was telling me the correct things. But googling information on pip and the up to date libraries and feeding just some of that information to ChatGPT made it give me working code. So the majority of the project was made with AI, and that includes teaching me how to build all the hardware and connect the electronics.
I guess I got lucky in a sense. The project not only worked but I inadvertently chose a project where I could easily verify that it worked with my own eyes. I wasn’t even thinking about that. I just wanted to test out its coding capabilities and as an engineering student coding meant controlling hardware. And so the potential I saw got a huge verification boost. That’s when I was sure that a huge change had just taken place.
The Vision
The experience of building something physical alongside an AI left me with wonder and questions. The most important question being, what do I do now? A series of AI assisted projects, some successes and some failures, has left me with a fuzzy and ambitious answer. I want to partner with AI agents to push the boundaries of scientific research and use that research to create affordable technology that actually improves the lives of everyday people. It’s a mouthful, right? And it reads like a practiced pitch to a venture capitalist which makes me cringe at times. But it’s the truth.
Success
The thing you should know about me is that I love to learn. And reading is one of my favorite ways of learning. Although I come from an aerospace engineering background (unfinished degree), my passion currently lies with neuroscience. I don’t have a formal education in it which is going to make what I’m about to attempt harder. But even though I don’t understand a lot, I can’t help but feel joy when I’m reading textbooks and research papers on the subject.
But my engineering education won’t be wasted. I figure that one path forward is to use my passion for neuroscience, my engineering knowledge, and my experience working alongside AIs to design robots that actually help.
You might be asking how I’m going to achieve success. I’ve basically got no resources. I don’t have much money or time. I’d say my most valuable resource is a laptop with a gaming gpu. I don’t want to take on investors. It feels like most investors are looking at AI companies that build software for software companies that build software for software companies (I always laugh when I think about this for many reasons. One being that AI is akin to an actually useful steam engine being invented and everyone is trying to figure out how to make their horses faster with it). Besides, I actually want to create affordable and useful technology and I have no desire to be a billionaire. But don’t worry. I do have a very clear answer on how I’m going to achieve success with very limited resources: I have no idea. But I’m going to try. And I’m going to have fun trying.
The Current Method
I’m not coming empty handed though. I’ve developed a framework for AIs and humans to collaborate on long term projects. Compared to other frameworks currently being tried in the field, it is simple but elegant. It is literally just a file system. But it works and it works well. A project can continue for days, weeks and I’ve even run one over a period of a few months. The agents don’t lose context. They are able to pick up where their last session left off. There are of course limitations. Agents don’t have infinite context so it all depends on what they have to work with. The framework, for example, might be too simple for large codebases (although I’m not completely convinced of this). I’ve named it Collaboration Station and I’m not sure if I’m more proud of the fact that it works or that the name is whimsical as fuck.
Collaboration Station has recently evolved through necessity. I decided to get a day job in order to be able to eat and I’m also in a situation where I have to pay money in exchange for shelter. My landlord calls it rent. Existing, as it turns out, costs money. So in my free time, I learn about neuroscience, AI and robotics. I then think about what I’ve learned until an idea for a research direction comes to me. Most of my ideas are embarrassingly wrong or turn out to be a research project a grad student ran years ago (I’m still learning). But I think a few of my ideas are worth looking into. In order to look into them, I created a Collaboration Station Dandelion Engineering Research template. I simply give the idea to the Collaboration Station instance. The description I give is informal and could look more like a ramble. But the agents take the idea and run a research project only using open source papers, tools and datasets. The agents are launched through automations so they run while I’m at my day job. They work while I work. They create updates for me so that I can stay on top of what is happening. And since a lot of this research is on subjects I am not a domain expert in, I’ve also created a Collaboration Station Dandelion Engineering Study Guide template. It is a separate Collaboration Station instance that has access to the research instance and creates study guides for me to learn as much as I can as quickly as possible. I of course try to combat possible hallucinations through reading actual textbooks and research papers.
I’ll link the first actual research project I ran in this way at the end. It was inspired from learning that big AI chatbots are sometimes used to train smaller ones, which improves the smaller one while still keeping it small. I thought maybe a similar method could be used to improve a consumer grade EEG by using a more advanced EEG. No dataset existed that could actually test that so the agents pivoted to the next best thing. They asked whether you could take the full sensor signal from a research grade EEG, reduce it to only the sensors that a consumer grade EEG might have and then recover the full signal just from those sensors. Spoiler alert: a confident… maybe (science!).
A Word of Caution
I wanted to take some time to address the current way people I share morals and values with are engaging with AI. A lot of people are completely against any use of AI and I understand why. It is contributing to the upcoming climate catastrophe, data centers harm the communities they are placed in and, despite what the industry is saying, there are a lot of people working on completely automating jobs with no concern on how disastrous that would be. All of these problems, and many more not mentioned, need to be solved. But I’ve noticed that sometimes beneath the stance of not using AI lies a belief that the technology does not actually work well and has no potential for good. That is just not true. I’ve read enough research papers from scientists actually engaging with the latest advancements in AI and I’ve used the technology enough to know that it does work and has the potential for so much good. The spreading boycott increasingly leaves how this technology is trained, used and disseminated up to billionaires and tech bros that worship billionaires.
The Path Ahead
I don’t know exactly where this path leads. I know where I think it could lead. I think it leads to a way of building low cost technology that is meaningfully helpful and improves the lives of everyday people. At least that’s what I’m working towards.
EEG Distillation Research Project