This is a post about AI and the iPhone Duo, but it’s also kind of not a post about AI and the iPhone Duo. It’s a post about how I identify barriers in my life and how I use technology to solve them.
Back in April, I started exploring agentic AI technology, basically when AI does things on your behalf. What started a curious little experiment slowly turned into accessibility tools which have been life-changing for me.
In my case, AI means local running AI models, things that run on my own hardware, fully private and under my control. The opposite end of things to the ChatGPT and Claude codes of the world
To make it work, I’ve ended up building my own custom miniature ‘platform type thingy’. This post is about how that works, how the iPhone Duo is part of it, and how it’s helping make my life better.
I built my agentic setup mostly so I could do things whilst lying flat. I have a spinal cord injury that means I need to spend a lot of time lying flat and if I don’t, I end up in lots and lots of pain. Additionally, back in August I sat in a chair badly and managed to hurt myself even more.
Easily 90% of the interactions I have with my AI are via my phone using the chat app Telegram. Essentially, I send it messages and it does stuff and then sends me messages back.
On a more technical level, the system’s got three parts.
As with many technical things, there’s a bit more detail than that, but we’ll get to that later.
Fundamentally, my agentic setup is fairly simple. I’m not doing anything particularly clever. It’s just a way for me to use my computer without needing to sit in front of it
Broadly speaking, I can split what I do into four categories:
Due to my injury and pain levels, I can only sit in my blue chair for a couple hours a day. The rest of the time I’m pretty much lying flat. To make this work, I’ve ended up building lots of little tools.
I use the agentic AI to build a tool, but once it’s built, it runs without any AI at all. It’s about as close as I get to vibe coding.
The tools mostly relate to my environment or health. For example, A monitoring tool for the solar panels on the shed that sends me graphs every hour or so. Tools to control lighting, heating and fans in my bedroom.
Some of these tools are also very accessibility focussed, for example, a tool that monitors how often I’m brushing my teeth by detecting my Bluetooth toothbrush, or tools that send me reminders and prompts to do with hydration and other health issues. I’ve also got tools which help me identify when sensory friendly clothes has gone on sale and things like that.
I’ve even built a tool that helps me monitor how long I’m laying flat to make sure that I lay flat enough… but not too much!
None of these tools are particularly clever, but they’re all very helpful and they help me live a life that is easier and less painful. All of this is done, laying flat from my phone via Telegram.
The second use case is mundane work stuff. I don’t have one job as such, I have three side projects in a trench coat pretending to be a job. I mix between crime fighting, neurodiversity stuff, accessibility stuff, and a little bit of everything in between.
I’m using Hermes to help me with admin tasks.
Things like finding invoices and receipts for my accountant & managing my project notes in Obsidian.
It also helps me write project outlines and other documents. The key point there is it helps me, it doesn’t do it for me. Every slide in every talk is made by me. The AI tools just reduce some of the barriers by helping me to sort and access my own information.
I’ve also ended up using a tool called llmwiki an awful lot. It really deserves a post of its own, but essentially it’s about building knowledge bases around a specific topic and then being able to query them.
I create a custom wiki whenever I’m researching a topic, doing a due diligence or just need a bunch of info in one place. I also created one about me. I ingested all of my blog posts, tweets, bluesky posts, podcasts, etc, and it pulled out all of the topics and concepts. It’s incredibly helpful, especially when I wanna find a good quote or reference something I did many years ago.
The third use case is custom user interfaces. This is a little bit different from custom tools in that most of these user interfaces are used for a single thing and deleted. The main goal is to remove some barrier to make something I’ve got to do easier.
For example, this year I took part in reviewing CSUN session proposals. The online portal for reviewing submissions was really hard to use. Especially when laying flat using my phone.
So I used Hermes to create a custom user interface. It was optimised for my exact phone, with big, easy-to-read text and large buttons. The custom UI allowed me to quickly work through submissions, reviewing them against the criteria and writing feedback. Once I’d finished the process offline, I then had an easy method to copy and paste my responses into the proper form.
The custom UI probably saved me half a day of being in pain. I could get it done lying flat, and I didn’t have to sit at my desk.
The third use case is agentic development. A large part of my life is writing software for other people to use. Whether it’s tools to identify financial crime or helping my clients with Research and Development projects… I write a lot of code that gets used by a lot of people.
I could literally write a short book about how I go about doing this, but the very short version would be that I use small local models to solve fairly straightforward problems. I’ve refined out the system, so it codes like I do. A little bit worryingly, the simpler the model I use, the more it codes like me, hmmm… oh dear.
Agentic development is sort of the opposite of vibe coding. I’m not just chatting to my AI and then throwing stuff into production. I’m following a structured engineering process, which happens to use AI to generate quite a lot of the code. However, everything is being reviewed and everything is being considered. I feel more like the conductor of an orchestra rather than being a violinist.
I get most of it done laying flat, which is remarkable. I’ve been building a new product over the last few months, which is probably the most interesting thing I’ve ever done. It might be the most impactful too. 99% of it was built agenticaly using small models running on my Mac or gaming GPU.
Less and less of what I do requires me to sit at a computer. It’s such a huge accessibility improvement I find it hard to put into words.
There’s a bunch of things I’m also NOT doing. I wrote about this before in my AI principles, but essentially I don’t have the AI pretend to be me. I don’t directly sell the output, and I don’t let it make decisions that affect anyone apart from me.
I also don’t really depend on the AI for inbuilt knowledge. The whole point of the agentic AI is that it goes out to find information as it needs it. Essentially, I treat anything and everything it tells me with a large dose of scepticism.
It’s sort of remarkable that this is all running on my own hardware under my control, none of this is using the cloud, none of this is boiling an ocean, it uses around £1-2 pounds of electricity a day, often from solar power.
Along the way, I’ve ended up experimenting with lots and lots of different hardware, both on the AI side and on the user interface side, here’s a quick recap and some thoughts on how the iPhone Duo fits into all of this.
On the AI side, there’s two sections. There’s where the agent lives and there’s where the AI bits run.
I have two ‘agents’. My work agent runs on a M3 Ultra Mac Studio, and my personal agent runs on an old HP 400 Mini PC.
The agent software I use is called Hermes and it more or less ‘owns’ the computer. It has access to anything the computer has access to, and it basically drives the computer using the user interface. This doesn’t really require much power, the cheap PC works really well. It was about £200 from a second hand store. The same PC hosts about half of my accessibility tools too.
The main reason my work agent runs on a Mac studio is because it regularly has to run very large test suites, which can be really slow on older hardware. Originally the Mac Studio pulled double duty and ran the AI models too, but I don’t really use it that way anymore.
I keep my personal and work completely separate for security reasons. It also gives me an easy way to test new things in a low risk environment.
The agents need an AI provider. That’s the bit that does the actual processing of the model weights. It takes a text request and returns a set of actions for Hermes to complete. It’s the bit than needs the ooomph!
I’ll write more about my set up in the future, cause it’s erm… specific… but essentially I’m running models across a gaming PC with a 5090, a workstation with a pair of 5090s and an M5 Max based Mac Studio.
The hardware sort of came together over time bit by bit. It all sits behind a proxy which does very clever load balancing so tasks get distributed efficiently. It’s running some custom code that I wrote because I got annoyed and, well, that’s a story for another day.
Believe or not, the M5 Max would be enough to get most of what I do done. However, to get things done quickly, I need to work in parallel, and that requires more hardware. Most of the time I have between three and five tasks running.
As I mentioned before, it uses about £1-2 pounds of electricity a day. I don’t have an exact number, but I’m using somewhere between 2 to 4 billion tokens a month. Based on open router pricing for the same models, we’re gonna break even before the end of the year. It’s also rather good for playing cyberpunk from time to time :)
I’ve used a lot of different hardware for my user interface. Aka, the thing I hold while I’m laying flat.
Early after my injury, I tried VR headsets and things like that… they work but they are very clunky & unreliable. I’ve also tried using a projector pointed at my ceiling, but I can’t read it well enough for it to work comfortably.
I’ve used an iPad mini and my iPhone 15 Pro. The iPad mini was ‘fine’ but it’s a bit too big & heavy. Way too easy to drop on my face.
All in I do about 90% using my iPhone 15 Pro. It’s light, easy to hold, and I can use it for typing into Telegram and reading the output very easily. The main thing I couldn’t do with my iPhone was code reviews. They required me to go sit in my big blue chair, which could get very painful.
For the last five days i’ve been trying out an iPhone Duo. It’s a really odd device to work with. When folded closed, it’s a very strange iPhone that’s weirdly wide and weirdly short. It also feels really heavy. It does work well though. I’d say I’m using it closed about 50-60% of the time. When closed, I think it’d be best described as clunky. My iPhone 15 is a better iPhone than the iPhone Duo.
However, unfolded, it’s a bit of a game changer. It’s light enough that I can hold it upright in portrait and type away without any problems. The large screen lets me do proper multitasking in landscape. I spend a huge amount of time swapping back and forth between the same few apps, having them side by side is really, really useful.
The ability to move from closed to open without changing context helps a lot with productivity. I’ll often start something with the phone closed and then open it up as I need a bigger screen and then close it again. It’s all very smooth and reliable. It just seems to work.
Having everything all in one place is probably the killer feature. I’ve tried running a iPhone and an iPad at the same time and it got very, very annoying.
It’s only been a few days and I’m already finding ways to use the phone to save myself from being in pain. For example, I’ve been using it for code reviews and it’s amazing. I think it’s actually easier to code review from the Duo than it is from my desk at this point, which is very strange. I strongly suspect I’ll use Hermes to build a custom tool to refine some of the interactions, so they’re just that little bit easier.
I got really lucky getting the Duo. We ordered it via the Apple Business team and it wasn’t due to arrive till January… then on release day the Apple rep called me to say she’d managed to secure one in store ready for collection. I tested positive for COVID the same day, so a friend of mine collected it and I’ve been playing with it since.
It’s sort of funny that in order to explain the iPhone Duo, I also have to explain how I use agentic AI.. but to me they’re all part of the same package. So far, the Duo has been the best user interface for what I’m doing. I don’t think I could recommend it to most people because it is clunky, weird and a little bit limited, but for me it’s pretty much perfect.
Wow, this post got long, and it could easily be five times longer! I’ve not really gone into much detail on anything. I’ve given a really high-level overview at best.
Technology is enabling things that I never thought to be possible. When I had my injury five years ago, I could only do about 20% of my life laying flat. These days it’s more like 80%. Tools like Hermes and the Duo are making this happen for me.
Both of these tools come with risk and complexity. Don’t get me started on what I have to do to keep Hermes secure… or all of the things I need to do to make everything work smoothly. It’s a bit of a pain in the bum still. I wouldn’t exactly recommend it for everyday users. I’ve been supporting a few clients to use the technology, and it can be hard-going. It’s very new, it’s very clunky, and it’s very unreliable sometimes.
I’m painfully aware how lucky I am that I can try these tools. I’m using them to run multiple businesses and they’re quite literally investments in my future.
I’m excited for technology right now in a way I haven’t been for a long time. New tools are enabling new things for me. They’re removing barriers that I never considered solvable, and my life is better than it’s been in years.
