I don't know where to begin. This is a long story, but it all started when I saw that DEV was having a ‘finish-it--up-a-thon’. I was instantly interested, with a particular project in mind. I’ve been working on govend on and off since 2024. It’s something I started in a web developer apprenticeship. When I first ‘launched’ this ‘minimum viable product’ I was so proud of it. Every line of ruby, written by me. - No assistance from AI. The nokogiri web scraper implementation, figured out all on my own. It was ugly and I had issues with dates not saving properly in the database with the scrapers. Only recently after deciding I was going to finish my project did I address this. If I am going to ship I refuse to ship slop if it is within my knowledge and power. This is the latest before and after shot I have here:
Before on the left, after on the rightI really gave myself an undertaking. I didn’t realize how much work I had left to do. Given other priorities I was sadly not able to participate in the DEV challenge. But it gave me the itch of motivation I probably needed. The biggest part of this challenge for me was to successfully self-host my app locally. I’m using AI and I did not have a personal 'server' on my local network. I had everything running off other services… render, neon, vercel, heroku, github pages.
I quickly realized that I had a problem when I wanted to use AI in my apps and allow users to use the AI. One of the issues I ran into is that AI cloud servers are expensive. Another issue I ran into is connecting cloud services to a local agent. In my govend app, I created a chat interface a while back where users can talk to ollama and ask about events during certain dates, or to find events by location. This was easy to do locally, I had ollama reach out to algolia with the users request, and bring back related events. But I want to do this… not on my personal computer.
I must admit all of this is a scary leap for me, but a necessary one. I looked into methods of affordable self-hosting and after some digging, I was convinced that the Jetson Orin Nano was the answer. It’s designed to handle AI tasks on small models. Good enough for my use case. And I was really liking how it was a one-and-done situation, no need to buy external little this-and-thats. I’m already doing a big new undertaking, don’t wanna complicate it. I’m not gonna spend $100 or whatever it is now a month on a cloud ai server - I’ll spend a few hundred and be done. (For now, evil laugh)
What I actually got:
So that little piece of information that pops up and says ‘enter yes to continue’, I never saw that during my first attempt. I kept retrying over and over, never seeing the screen I needed to see.
I’ll tell you why:
At some point, I finally got the thing to boot by plugging in the orin nano and pressing the button on the monitor at the exact same time. Things started , packages zipping across the screen, general excitement. And then, I waited. For like, half an hour. Is it broken? What is happening? General panic.
After what seemed to be rough 45 minutes to an hour, I pulled the plug. Something didn’t seem right. I carefully plugged in the nano and pressed the monitor power button at the same time. It booted, and asked me for a login. Login? What login? I haven’t made a login? That usb became read-only afterward, so I couldn't reformat it normally. I ended up using a different usb for the installer.
Now armed with the knowledge of the scope of my personal problems, I create a new usb, ready to battle with peripherals. It works correctly when you both know what your problems are and how to handle them.
Got a mini hdmi cable so I could get rid of the obnoxious pile of a screen I was using prior. Way better!In the days (weeks?) that followed, I got everything setup. I learned how to self host my own apps on the nano. Having cleared out all of my heroku’s and render’s I had a small pool of domain names at my disposal as well. I spent a few days really trying to get govend to a proper state so I could share my app and help vendors and artists find events to vend at and manage their events in one place. As a vendor of 5 years I know this is needed. Small businesses are the wild west and don’t have too many specialized tools. I really wanted to see it come alive. I could spend forever on it, but I have to make it available at some point.
I also cleaned up my flippy card app that uses ollama for the generation of flashcards, a memory game, and self quizzes. I am able to use a single ollama instance for both of my apps. How awesome is that? And basically, since my database is separated from supporting services, my web server is basically just a little connecting hub for apps and services. Pretty cool! I think its relevant to insert here that AI can be a security nightmare if you don't button stuff up.
I learned about securing things, using proxies, and closing unused ports. I asked a couple trusted people to take a peek at my app and I had someone run a scan on it before sharing with their bot as a security measure. I openly admit I had not considered the ai security too heavily, and things were found and addressed. It’s interesting to be fairly confident about an app and then AI throws a wrench into everything. I did not anticipate all the time that has to be spent on security. Massive learning experience, and had me groaning at my server towards the end.
I installed claude code and used it to help me build out a server ‘hub’ page quickly, where I share some live apps, blog posts, and relevant links. After a carefully chosen prompt it built quickly with my aesthetic in mind. I proceeded to tweak it, adding a carousel to the projects section on top so it would look different from the bottom blog section. I pulled in code I used in previous projects to let Claude know what I was trying to do. Having built a bunch of stuff prior, it made constructing this page incredibly simple.
Look at the tiny server go!I would have to say a large part of cleanup involved making sure everything is secure as possible.
What do you think?