Linux for Embedded Development

A little over a year ago, I bought a new Casio FX-9750GIII graphing calculator. It really appealed to me for several reasons so I decided to buy one even though my days of collecting calculators are mostly over. One of the main things I like is that it comes with Python which has become one of my favorite programming languages. Another interesting thing is that the processor speed was doubled to 59MHz over the preceding FX-9750GII making it as fast as the FX-9860GII and the FX-CG50. Amazingly, there is a port of Xcas for it, so you can run the same Computer Algebra System (CAS) used in the HP Prime, TI-89, and TI-Nspire. The styling of this one has changed back to the standard rectangular design which is much better than the awkward and ugly rounded design Casio used for some of the preceding models. All of this is very impressive and it only costs $39! Best of all, it can run programs written in C and assembly like previous Casio models. Other modern calculators like the HP Prime, TI-Nspire, and TI-84+ CE don't allow native code execution which is why I've stayed away from them. The catch with the FX-9750GIII is that the GCC toolchain put together by the community only runs on Linux. It took quite a bit of work getting the toolchain installed on Ubuntu in a virtual machine since I didn't have much experience with Linux. A few months ago, I moved everything over to an old server that someone donated. Below are some things I've learned while doing more and more of my embedded projects on Ubuntu.

Virtual Machine
At first, I installed Ubuntu Server on VirtualBox so I could use the GCC toolchain for the FX-9750GIII. It took quite a while to install and then some more work to get things like networking and a shared folder with the host to work. The GCC toolchain turned out to be very problematic to install which was not entirely due to my lack of experience with Linux. Thankfully, the maintainer of the toolchain put in a lot of work helping me get it working on the Planet Casio forum. One of the first problems was trying to get the scripts working without understanding how the various types of shells work in Linux which is very different from the Windows command prompt I'm used to using. There is a package called GiteaPC that the maintainers put together that is supposed to install everything, but this didn't work for me at first either. Other problems were beyond my control such as not being able to update CMake to the version the toolchain requires. For this, I had to update Ubtuntu from 20.04 LTS to 21.04 which does not have long term support. CMake also had problems finding the PATH variables. Eventually, I did get the toolchain running, though it failed when trying to install it again sometime later since the maintainers had changed the toolchain's structure so it no longer built correctly. In the end, I did get GiteaPC working, again with the help of the maintainers, though I have to say the weeks-long process of just getting to Hello, World was pretty awful. All of this led me to reinstalling Ubuntu several times on VirtualBox.

Running Ubuntu in VirtualBox worked pretty well, and I started exploring other things I could do with Linux. Using it for everyday tasks seems like the best way to get good at it. One of the first things I set up was a Python script to collect and show traffic from my IRC client. Originally, I set this up under Windows after freenode collapsed since I needed to switch IRC clients to connect to Libera. The new client, HexChat, didn't have a digest window showing me all the traffic in one place from all the channels I care about, so I made a Python plugin for HexChat to relay that traffic to a Python script listening over TCP. The script worked with almost no changes on Ubuntu, so I switched to using that to monitor IRC traffic. Leaving the virtual machine running minimized all the time so I could monitor the traffic was convenient but gobbled up a lot of resources even after adjusting how much RAM and how many cores VirtualBox could use. It was most noticeable after my laptop woke up from sleep when the whole system would be sluggish and the fans would run at full speed. Even the mouse pointer would freeze in Windows for 2-3 seconds at a time while VirtualBox was minimized, so I started investigating other possibilities for running Linux.

Linux Server
Since VirtualBox was too slow to keep minimized (even Ubuntu Server with no GUI), I tried setting up Ubuntu to dual boot with Windows. It took several tries to get GRUB to work right. Part of the problem was that the Optane feature of the SSD in my laptop needed to be disabled. Interestingly, this caused a separate 13GB drive to show up in Windows since it was no longer being used by Optane for caching files from the SSD. There doesn't seem to be any way I can find to merge the drive back and re-enable Optane even after removing GRUB and Linux. In the end, it was too inconvenient to shut down Windows and boot Linux every time I wanted to use it, and there were too many things on Windows that I couldn't live without to let me switch completely over to Linux.

Next, I started looking at cheap ways to get a computer I could use as a Linux server that I could log into over SSH. The Intel NUC computers were appealing since they are so small, but they are surprisingly expensive. When I asked at one of the online programming meetings I attend about what type of old computer I should look for, one of the participants offered to give me an old machine he had. It was quite a surprise when I got it home to see how nice it was: an HP Z230 workstation from 2014 with 32GB RAM, 4GHZ Xeon processor with 8 cores, a 256GB SSD, and a 1TB hard drive. Installing Ubuntu Server on this one was much faster than installing it on VirtualBox.

Connecting over SSH
After installing Ubuntu, I set it up to accept connections over SSH so I could connect from my Windows laptop with PuTTY. With that working, I took down the monitor, mouse, and keyboard and kept the server under a table in my room since I could do everything I needed over SSH. This setup worked fine for connecting with my laptop while on my home network, but there was no way to connect from outside, so I opened a port for SSH in the router firewall and mapped my network's IP to a domain name with DuckDNS. This setup worked really well.

A few months ago, I moved to a new place and had to reconfigure the server. Since I didn't have access to the new landlord's router, there was no way to open a port in the firewall. Also, the router prevents wireless devices from communicating with wired and wireless devices on the same network as a security measure, so my laptop couldn't communicate with the server even when I was at home. To get around this, I set up ngrok so it would forward SSH traffic from the public URL they offer to my server. The free version of their service randomizes the public URL and port number, so I used a cron job to map the randomized URL to my DuckDNS domain name. The ngrok daemon allows you to query a local port for a JSON object with the URL, port number, and other information, so it was pretty easy to pass the information on to DuckDNS with a Python script. For the randomized port, I used sendmail to have the cron job email me the port number. As long as the server doesn't go offline, the port doesn't change, so I can connect to the server over SSH with my laptop whether I'm at home or somewhere else.

Backups
On my Windows machine, all of my projects are saved on Google Drive so they are backed up to the cloud automatically. For the Linux server, I wanted to set up something similar for my projects, but there doesn't seem to be any way to install it for command-line only Linux. Eventually, I found rclone which lets you synchronize files between your local machine and cloud services like Google Drive and Amazon AWS. A cron job runs at 4am everyday to zip my home directory, upload it to the cloud with rclone, and make a second copy to the local hard drive with rsync. The first version uploaded the backup to Google Drive. The Google authentication is a bit inconvenient since permission for rclone needs to be enabled specifically in developer options, and in my case the permission expired after a few weeks. For a while after permission expired, the script ran everyday but nothing was backed up to Google. Next, I switched to Dropbox which has worked without any problems since then.

Linux tools
Here are some tools I've gotten to know about as well as some things I devised myself to make life easier:

tmux - This terminal multiplexer is one of the best parts of using Linux in a terminal. It's easy to set up new windows for anything I'm working on, and I have a default setup that loads from a script. The best part is that the session stays loaded between disconnects so it's easy to pick up where I left off every time.

vim - At first, I used nano as my text editor, though I switched to vim when it became clear that nano is not ideal for editing source files. So far, I've been able to get pretty good at it without having mastered it yet. Editing the configuration file to enable some important features like going to the previous line when pressing backspace on an empty line and adding proper indentation go a really long way to making it comfortable to use. The separate editing and insert modes are interesting, though I'm still suspicious that it comes out to less keystrokes in the end than something like Notepad++ on Windows.

scripts -  In my home directory there is a "scripts" directory that can be accessed from anywhere. It's really convenient to put bash and Python scripts there to do things I do often like starting my tmux setup or starting the IRC monitor mentioned above. There are two scripts that create new bash or Python scripts based on a template which has the appropriate shebang and other stuff I commonly use. One of those things is the definition of a printc function that takes a color as it's second argument, so it's easy to add colored text to scripts in either of those languages. Another really useful thing is a script called warp that offers a menu of directories I often use. Invoking the script with a . before it causes the change of directory the script performs to propagate up to the shell that called it. With this, I can quickly jump between directories without typing out long, multi-level paths.

Embedded Development
Here are the tools I've been using for embedded development:

GCC - It's awesome to have a C compiler so easy to use at the command line after coming from Windows where things are a bit less convenient. So far, I haven't made any substantial projects with this, but I have used it to link x86 assembly programs so they can use C library functions. In the tmux setup mentioned above, there is a temporary C file open in vim so I can try out short snippets.

NASM - This is the same x86 assembler that I used for macro expansion in my Robot Game project comparing 6502 programming languages. On Linux, I'm using it to assemble x86 assembly for a project that I plan to post about soon.

mingw32 - This is a C cross-compiler that produces executables for Windows. Like GCC, it's easy to compile and link a program with one program. Wine runs the result so you can test console programs in the Linux terminal. I'll definitely use this over Microsoft's C++ compiler if I need to compile a console C program in the future.

6502 - There are several neat things on Linux for 6502 programming:

• The CC65 C compiler runs natively, so the excellent CA65 assembler that comes as part of that is available. 
• Macroassembler AS, which I'm using for my current 6502 project, is Windows only on the other hand. It runs under Wine in the terminal.
• The Calypsi C compiler is a Linux only project like the toolchain for the FX-9750GIII that got me into Linux in the first place. In some short tests I've done, the output seems really good compared to CC65.
• The calc arbitrary precision calculator program is not specific to the 6502, though it was really useful for generating tables for my 6507 Calculator project.

GCC for MSP430 - Getting this working was a little less straightforward than the GCC for Linux. Installing msp430-gcc with apt gets you the 4.6.3 version from 2013. Since then, a completely new version has been created which TI hosts on their site. By default, it tries to install to /root/ti, though I was eventually able to change this. This is the version I used for my Kitty Salon project. Copying the compiled binary to Windows with PuTTY's pscp utility takes less time than flashing it to the MSP430.

Python - Like with GCC, I have a temporary file open in my tmux setup to try out short snippets. A lot of my housekeeping tasks are in Python since I know it better than bash. The main project I'm working on for the 6502 is in Python, so I use it a lot nowadays.