Voice Controlled Web Apps Pt. 1 (Windows)

NodeJS, Client Side Javascript, and Google Speech API


This series will introduce newcomers to web programming using a few of the latest tools available. I’ve dabbled in building web sites since Microsoft Frontpage 2003 came out, but always ran into frustrating inconsistiences (casualties of the browser wars) and intimidating combinations of programming languages to start a server and handle user interactions (PHP, AJAX, HTTP POST and GET, yuck).

Recently I’d been hearing a lot about Node.js, software for building web servers with javascript. When it was introduced at JSConf in 2009, it received a standing ovation. When was the last time you heard of a new technology receiving a standing ovation? With all of the hype surrounding NodeJS, I decided to dabble again and was swept off my feet by the fact that I only have to know one language. Javascript is used to create the web server, and javascript is used to make the web page interactive. With added libraries, we can use javascript to control lights, relays, and motors with the use of Arduino, Raspberry Pi, Galileo, and others, and we can use the relatively new HTML5 web speech components to control all that hardware with voice commands!

Part 1 of this tutorial will walk you through installing the bare minimum on a Windows system (linux and MacOS instructions will come later) and serve up a ‘hello world’ HTML page.

Part 2 will introduce client side javascript — that is, programs that run in a user’s web browser to make the page interactive. We’ll start with a button that changes the background color.

Parts 1 and 2 could be switched, you could start with html and then learn how to start the server, as long as you do both before part 3.

Part 3 will use the excellent Johnny-Five library (meant for programming robots with javascript) to allow our webserver to control an Arduino. It could just as easily control the GPIO of a Rapsberry Pi, BeagleBone, or Galileo. Socket.IO will be uesd to send messages from the client to the server.

Part 4 adds voice control to the mix, explaining how to use the Annyang library to execute javascript functions (and therefore control attached hardware) within the web browser. I’ll also cover creating an SSL certificate here so the browser listens continuously without timing out.

Part 5 moves all of this functionality to the Galileo platform, which can run our web server and interface directly with other hardware, via digital I/O and serial communication.


Without further ado:

Install the Basics

Visit the official NodeJS website and download the latest. I’ll be using 4.2.2, hopefully they won’t change too much by the time you read this.

Pick out a good text editor: I’ll use Komodo Edit because it’s open source and lets me use Vi keybindings, but Notepad++ and Sublime Text are popular. Any of these will give you helpful features like syntax highlighting and auto-completion which make programming a million times more pleasant than trying to use notepad.

Go through the Next →Accept →Next →Next →OK →OK →Finish rigamarole of Windows installations to get NodeJS installed.

Next we’re going to do a javascript hello world. NodeJS is simply a program that executes javascript files. We’ll write a command in a javascript file, and tell NodeJS to run that file for us. This first program is just one line of code:

console.log("Hello World");

Type that into your text editor, and save it as something like ‘firsttry.js’ or anything you want as long as it has that .js file extension.

Next, open Powershell (you can hit start and just type ‘powershell’ and hit enter). This is a program you might never have touched, but it comes with Windows and has a lot of powerful features — it’s how we’ll control the webserver. But first, try a few commands, see what ls, cd, and pwd do. They stand for List, Change Directory, and Print Working Directory.


Once you’ve navigated to the folder where you saved your first js file, you can type node followed by the filename. This uses NodeJS to execute firsttry.js, which simply prints “Hello World” to the console — another word for the command line. We’ll use console.log() a lot, it can give status messages from the server running in the command line, and it can also give status messages in the web browser, which has its own console we’ll see later.

So that’s a pretty boring program, but it’s important to make sure that ‘hello world’ works if we expect to run a web server on our machine.

Of course, there are whole books written on and whole careers based off writing and running web servers. For our purposes, I want to provide this working model: a web server is a program that listens for requests, and fulfills certain requests however you tell it to. The ‘listening for requests’ is mostly handled for us, we just have to write code to respond to them.

The NodeJS documentation provides dozens of functions for this purpose, but doesn’t give a simple example. I’ll use a stripped-down example from this awesome instructable to get us started. You could copy paste this example code into your editor, but I recommend typing it in yourself to spend some quality time with each component.

https://gist.github.com/jazzyjackson/6ae764d24ead285f9ba1

If you haven’t coded much, this is a little crazy, but thankfully we don’t have to come up with this ourselves. This chunk of code actually handles quite a lot: it uses require() to import or include the ‘http’ and ‘fs’ (short for filesystem) features of nodejs. It creates a server object and starts listening on port 80. It answers any request by sending index.html back, and delivers a nice error message if it has any trouble.

To get your first website up and running, save this javascript file, and create another file that contains nothing but the phrase “hello world.” Or, if you want to work ahead:

<h1> Hello World! </h1>

console.log("Hello Console!")

Save that file as index.html in the same directory as simpleserver.js and go back to Powershell. Remember how to use node to execute a file? Try it on the javascript file containing the web server code.

The very first time you run javascript code that creates a server, you’ll get the following message (unless you have Windows Firewall disabled):


Make sure “Private networks” is checkmarked before clicking Allow!

I checked Private and Public, since I want to try serving webpages on networks besides my own. Windows will only ask you once! If you don’t allow both networks now, it’s a bit of a pain to change the settings later.

Once you allow NodeJS to communicate, you can open a web browser and try it: just type in ‘localhost’ in the address bar.


If you added that console.log(“Hello Console”), you get to check out the web browser’s console. That line of code executes inside Chrome and writes its message to Chrome’s console, found in the Inspect element menu. We’ll use this a lot later on, so now’s a good time to find it.

That’s all for part one. Next up is writing our own javascript (don’t worry, it won’t be as complicated as the server function, at least at first 😉

References:

http://www.instructables.com/id/Javascript-robotics-and-browser-based-Arduino-cont/step6/Create-a-web-interface-with-Bootstrap/
http://www.instructables.com/id/Javascript-robotics-and-browser-based-Arduino-cont/step6/Create-a-web-interface-with-Bootstrap/
http://www.instructables.com/id/Javascript-robotics-and-browser-based-Arduino-cont/step6/Create-a-web-interface-with-Bootstrap/

And NodeJS Documentation for FileSystem and HTTP.

Preparing FFMPEG for free screen capture, time lapse, and GIF making.

Debian

Put together some info from here and here and here.

Add deb-multimedia non-free to your repo and install ffmpeg with the following commands in your terminal:

su
echo deb http://www.deb-multimedia.org jessie main non-free >>/etc/apt/sources.list
apt-get update
apt-get install ffmpeg

That’s all! You’re ready to make screen captures and gifs.

Windows 7/8/10

Download and Install

First, install WinRAR or 7zip so you can open 7z compressed files.

Download the latest ffmpeg.

Adding ffmpeg to your ‘PATH’

Add ffmpeg to your path so it can be executed anywhere. There will be only cosmetic differences between Windows 7 and Windows 10.


I found that one one version of Windows I was unable to edit the path if I simply hit Start and searched for “system environment variables” — instead I had to open the dialog via System (control panel) and then click “Advanced System Settings” and then “Environment Variables”. I guess by clicking on the ‘Advanced system settings’ with the little security icon it gives me the admin privilege I need to change the settings.


After clicking “Environment Variables…” You’ll get a list of user variables and a list of system variables. Scroll through the System variables until you find “Path” and click Edit, you’ll get the “Edit System Variable” dialog. Here each folder containing executables is listed in the format C:foobar;C:foobar; — that is, full file path, ending in a backslash, seperated by semicolons. So we’ll add our ffmpeg folder extracted from the 7z file, specifically, its bin folder. I dropped the whole thing in to my C drive, renamed the long folder name to just ‘ffmpeg’ and added the following to my ‘Variable Value’ (highlighted in the screenshot). The backslash at the end is important.

;C:ffmpegbin


After OK OK OKing out of there, you should be able to start powershell and start ffmpeg from anywhere. if you get the following when typing ffmpeg into powershell, then everything worked!


This stackoverflow discussion was helpful in figuring out how to do screen capture on Windows. I ran the following command to grab my Powershell window at 10 frames per second and saving to the file out2.mov. Here is the official documentation which details a few options.

> ffmpeg -f gdigrab -framerate 10 -i title=”Windows Powershell” out2.mov

I wrote a more thorough guide to screen capture here:

https://medium.com/p/3aeddbeb161b

FFMPEG is handy at converting video to gif, too!

> ffmpeg -t 2 -i out2.mov firsttry.gif

Pretty neat!


I wrote a guide to high quality gif making here:

https://medium.com/p/3aeddbeb161b

References linked:

https://medium.com/p/3aeddbeb161b
https://medium.com/p/3aeddbeb161b