Easy Come, Easy Go

Dang! Haven just written about the wonders of both open source and free services, I came face to face with a downside. What’s given freely in an API can just as easily disappear. I’m about 1/8th of the way through two online courses from DataCamp on using python to analyze financial data, only to find out that Google and Yahoo have both dropped (as of late last fall) their historical price data API (Application Programming Interface) that everything depends on. The key tools, an open source program called pandas, has been updated with work-arounds, but the course material hasn’t been updated, leaving me dead in the water. The data was provided free by both Yahoo and Google for decades, so while they have no obligation to keep providing the data, it has a pretty wide-spread impact when they stop. Lots of dependent applications broken!

monkey shoving laptop off of desk

Open Source + Free Tier Services: A Cornucopia for Hobbyists

Open Source logo I am constantly amazed by the amount, variety, and quality of open source software that is available, as well as the free (for small volume users) commercial Information and Communications Technology (ICT) services that exist. Open source has, in many domains, proven itself to be a reliable and cost-effective means of developing and improving software. I think an economic surplus for commercial companies has also contributed to this movement, as it has contributed to the free services that are available. (Although no doubt the free services are also intended as a gateway to the companies’ paid services.)

A little application I wrote the other evening provides an excellent example of this in action. The application is a software program that once a day checks inventory at our closest liquor stores for a certain hard to find and rarely in stock bourbon. If that particular bourbon is in stock at either store, it sends out a text alert. I wrote the application in Python, an open source programming language. It’s among the most popular programming languages in the world (and almost all its competitors are also open source). It used to be that companies paid large sums of money for commercial language compilers, while companies like Microsoft and Borland sold programming languages for PCs at hundreds of dollars per copy. Now programming language software is available for free, almost all as open source, while many software development environments are also free, and many open source.

The total volume of code involved in this application to open a web page, render it, scrape it for content, and send a text message with the result is huge, yet I only needed to write 34 lines of code! How? By using open source libraries that others had developed. These libraries (reusable modules with well-defined interfaces) did all the heavy lifting, I just needed to write some code to glue them together in a certain way and provide the key parameters. There are currently over 100,000 third party libraries available as free as open source in the Python Package Index repository. Cost to use any of these packages? $0.00. Similar libraries exist for other software languages, such as PhP, C, and Javascript.

As mentioned above, the program sends out a text message if the inventory is greater than zero at one of the local stores. How does it do this? Well, it uses the free twilio.rest library provided by Twilio. Twilio provides automated SMS (text), voice call, and other telecommunications services that you access through your own software programs. They are a commercial company that charges for their services, but you can test it out for free, and according to some folks, very low volume users don’t seem to ever exhaust their free account. This is an example where hobbyists like myself can, at no cost, take advantage of services developed for larger commercial customers.

At first, I ran this application on my PC at home, using the Windows Task Scheduler to have it automatically run each morning. But a) the Windows scheduler is both a pain and a bit flaky and b) it won’t run if my PC is powered off. So, I ported my application over to run “in the cloud” on Amazon Web Services, using what Amazon calls their Lambda service. As Amazon describes it, “AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running… upload your code and Lambda takes care of everything required to run and scale your code with high availability.”  As with Twilio, this is a commercial fee-for-service offering, except that you can use it for free if you call your services less than one million times per month and use less than 3.2 million seconds of compute time per month.

Finally, all those servers at Amazon need to run an operating system. What operating system do they use? Mostly Linux. While Windows and MacOS may dominate the PC world, the overwhelming majority of servers in the world run Linux. And like Python, Linux is free and open source. Linux is also at the heart of the Android operating system used on many phones and can be found running smart TVs and the infotainment systems on many cars.

Software is fundamentally different from hardware, so you can’t fully duplicate this environment for physical things (although there’s also a much smaller Open Hardware movement that open sources the design of hardware, rather than patenting it or keeping it a trade secret).  Nevertheless, it’s amazing how much can be done solely with the combination of open source software and free services!

Some Christmas Entertainment from Yorick

A little bit from Yorick for the holidays. As I mentioned in a previous post, Yorick now responds to “Alexa” using Amazon’s Alexa voice, and responds to “Yorick” using a voice with audio effects added to better match his appearance. I used both in this edited sequence.

Whatever you celebrate this time of year, a happy holiday greeting to one and all.