After you installed JupyterLab on your Ubuntu Analytics server, you’ll realize that it becomes quite annoying after the first 10 times to start JupyterLab through ssh. It’s much easier to have it run when you start your server.

There are a couple of files you’ll have to create and run some commands to get this to work.

1. Create an executable bash script. Lets name it jupyterlab.sh and place it in your home directory.

   cd /home/username/
   sudo nano juperterlab.sh

2. Type in the same command that you use to start JupyterLab. Replace the <installpath> with the absolute path to your installation. For example no using ~ for home directory. Use /home/username for the absolute path.

   #!/bin/sh
   /<installpath>/anaconda3/bin/jupyter lab --ip=192.168.1.2 --port=12345 --no-browser
   

3. Make the file executable by typing the following

   sudo chmod +x jupyterlab.sh

4. Create a service file ending in .service. I use the nano editor but you can use any editor.

   sudo nano /etc/sytemd/system/jupyterlab.service

5. And type in the following details into the file

   [Unit]
   Description=Jupyterlab
   After=syslog.target network.target
   
   [Service]
   User=username
   ExecStart=/bin/bash /home/username/jupyterlab.sh
   
   [Install]
   WantedBy=multi-user.target

6. Save the file and then run the following command to enable the service to autostart.

   sudo systemctl enable jupyterlab.service

7. Start the service

   sudo service jupyterlab start

8. Navigate to the URL http://192.168.1.2:12345 and check if you can log in.
9. If this works then reboot the server and try the step above again.

   sudo reboot

10. If it works, Great ! If it doesn’t then try this command to see any issues.

    sudo service jupyterlab status

     

Installing Python

The easiest way to get not only Python but all the python libraries needed for data science work is to download the Anaconda distribution.
Detailed instructions are available at https://docs.anaconda.com/anaconda/install/linux/. I prefer to use Python version 3 but the instructions show you how to download Python 2 too.

There is also a choice between anaconda and miniconda. Miniconda as the name suggests, is a mini version of Anaconda that only contains the conda package manager and it’s dependencies. If you prefer to have conda plus over 720 open source packages, install Anaconda.

Installing PostgresSQL

You will definitely need a database for your data science projects. At some point working on really large datasets is more efficient within a database. I chose to go with PostgreSQL 12.

Installation instructions can be found here: https://www.postgresql.org/download/linux/ubuntu/ but I’ve also copied the instructions for Ubuntu 18.04 LTS.

1. Create the file /etc/apt/sources.list.d/pgdg.list and add a line for the repository

   deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main

2. Import the repository signing key, and update the package lists

   wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key addsudo apt update

3. Finally install the application

   sudo apt install postgresql-12

Changing the location of the database to your SSD

If you remember how I built my Ubuntu Analytics server, I added in both an SSD (Solid state drive) and HDD (Hard Disk Drive) for storage. The idea was that my database and any other application that needed fast data read/write capabilities would sit on the SSD. Since Postgresql would be handlong a lot of data it became an ideal candidate to move to the SSD. There is good news that the it has a parameter called data_directory that just needs the folder path to your SSD
Digitalocean has a great tutorial on just that.

Once that’s done your database should be very fast.

Install JupyterLab

I found a good set of instructions on this website https://agent-jay.github.io/2018/03/jupyterserver/

I did make one change to the instruction. To use jupyterlab at localhost:ipaddress you will have to create an ssh tunnel. That’s what the last set of instructions was doing

ssh -N -f -L 8888:localhost:9999 user@domain.com #Change the specifics as required

Instead I started jupyterlab with the ip address of my server itself so i could reach it directly without using localhost. So if your headless Ubuntu server was at 192.168.1.2 on your LAN then you can use the command as follows:

jupyter lab --no-browser --ip=192.168.1.2 --port=12345

You can now access Jupyterlab at https://192.168.1.2:12345 without opening an ssh tunnel to the server.

Another thing to do if you’re getting a 403 error when you try to open a notebook could be because of folder restrictions. So make sure to change the default notebook directory location in the config file to a location you have write access to.

c.NotebookApp.notebook_dir = '/absolute/path/to/folder'

Updated: Mar 26, 2020

This post will walk you through getting your analytics server based on Ubuntu Server OS up and running.

Building the system

You’ve already seen how i chose the components for by build. Now putting them together is quite easy for someone who has built PCs many times before. But for someone who’s doing this for the first time it can be a little intimidating. So to help I’ve linked a video below that uses many of the components in my build and walks through the process. Of course it’s not exactly the same components but it’s close enough to help.

A couple of things to remember when you build:

• Static electricity can affect components. So touch some metal to discharge any current before you touch the components. This is especially important if you’re building on carpet and the humidity in the air is low like in winter.
• All the components are standardized. Which means that RAM modules across all manufacturers will fit on a motherboard. Of course you will have to make sure that your motherboard supports that type of component but once you do, components with the same spec from multiple manufacturers will fit. The reason i say that is these components will fit into the motherboard in only one way. So if a component doesn’t go into the motherboard easily then check if you’ve aligned the components (notches etc) correctly. Don’t try to force them in.
• Air flow across components are vital to keep them cool during operation. Try to keep the cables in your case as tidy as possible. It does help.

Installing Ubuntu Server

Ubuntu comes in multiple flavors. I decided to go with the server edition because I intended for it to be headless a.k.a no monitor or keyboard. I would connect to it using my laptop running Windows 10.

You can use any version of Ubuntu but go for it’s latest LTS (Long Term Support) edition. These LTS editions are supported for 5 years from launch.

As you can see it’s currently 16.04 but I decided to install 17.10 which is a standard release (Released in Oct 2017, hence 17.10). The reason i did that was kernel support. With the new Ryzen CPUs we will need an updated kernel to manage the CPU otherwise there are chances it would crash. I could have installed 16.04 and updated the kernel but i chose 17.10 which had the updated kernel as part of the base install itself. I also plan to upgrade the LTS version 18.04 a few months after it comes out in April 2018.

I’ve included a few links below to articles that will walk you through the install.

1. The official Ubuntu install tutorial: https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-server#0
2. Another step by step install guide for Ubuntu server 17.10: https://www.howtoforge.com/tutorial/ubuntu-minimal-server-install/

Once you’re done with your install you should be at a similar screen on your monitor.

At this point you should have your monitor and keyboard connected.

Setting up a static IP address in Ubuntu

So why do you need a static IP address? Usually when you connect your device to your network (wifi or wired) the router assigns an IP address based on what’s available. This means your computer could have different IP addresses on different days. So every time you wanted to connect to it remotely you would have to figure out what the IP address was. To keep this from happening you’ll have to go through and change some settings.

One thing you should do before setting up a static IP is to prevent an IP address conflict. Here’s what could happen: When you check for wifi networks at home and connect to your network, router assigns an IP address to your machine so it knows where to route network (including the internet) traffic. So let’s say your phone connects and gets the IP address 192.168.1.35 and you’ve also configured the static IP address to the same value. That would result in a conflict and you may not be able to connect to your server.

Depending on your router you may have to go to the admin page and find the section that lists the IP address ranges your router uses to assign to connected devices. You’ll see something similar to “Starting IP Address” and “Ending IP Address”. Leave the Ending IP address as is. Assuming your router is accessed at 192.168.1.1, change the IP address to start from 192.168.1.100 instead of 192.168.1.2. Save and reboot the router. This will make sure that any device connecting to your network gets IP addresses from 192.168.1.100 onwards. So you can use IP addresses 192.168.1.2 to 192.168.1.99 for static IP addresses.

Here’s an example for Netgear’s Nighthawk router: https://kb.netgear.com/24089/How-do-I-specify-the-pool-of-IP-addresses-assigned-by-my-Nighthawk-router

Once you’ve changed the starting IP address on your router you can proceed to setting up the static IP. One thing that has changed from Ubuntu 16.04 to 17.10 is the way you set up a static IP address. Follow this link to learn how to set it up. https://websiteforstudents.com/configuring-static-ips-ubuntu-17-10-servers/

Configuring SSH

Setting up ssh is needed for you to login to your headless server from your laptop. Setting it up is straightforward enough. Just type in the following command and you’re all set.

sudo apt install openssh-server

Once this is run the main thing to do is change the default configuration so that you can make it more secure. First type in the following command. This assumes you have the nano editor installed.

sudo nano /etc/ssh/sshd_config

There are a lot of configurations you can change but there are 3 changes i will recommend you do. Add/update the following lines in the file.

1. The first is to change the port from the default one that is 22. Let’s say you want to change it to port 12345. Change

   #Port 22

   to

   Port 12345

2. Make sure only certain users can ssh into your server. Let’s say your username was myname. Insert the following line

   AllowUsers myname

3. Prevent root from ssh access

   PermitRootLogin no

There are a lot of other settings you can add to harden your ssh security. I found a good list here: https://devops.profitbricks.com/tutorials/secure-the-ssh-server-on-ubuntu/#secure-the-ssh-configuration-file

Finally restart the SSH service by typing in the following

sudo service ssh restart

Configuring a SSH client

By now you’ve set up all the things needed for your Ubuntu server. The final step is to connect to the server using a SSH client. The usual recommended option for a SSH client is Putty. But I’ve found MobaXterm home edition to also work very well for my needs.

You can download them here:

• Putty: https://www.chiark.greenend.org.uk/~sgtatham/putty/
• MobaXterm: https://mobaxterm.mobatek.net/download-home-edition.html

Remember the static IP address and Port you set up earlier? You’ll need those now.

1. Type in the IP address
2. Enter the port number
3. Make sure SSH is selected
4. Hit Open

This should connect you to the server and prompt you for your username and password. Before that it may put up a message asking if you want to accept/store the server key. Say yes.

Once you’ve confirmed that you can login then you’re all set. You can disconnect your monitor and keyboard from your server.

Congratulations! You now have a working headless Ubuntu Server!

One thing is true about skills. They dull over time if you use them less often. I’ve been programming as a hobby, at school and at work over the last 15 years. But recently I’ve not been able to get very hands on. I’ve been lucky to have done well at work and moved to roles that manage analytics projects rather than working in the trenches. But a recent project at work gave me a chance to dust off my technical skills. The success of the project was due in large part to my ability to combine my team’s domain knowledge with my programming skills  to bring a complicated model to life. It was a good refresher of my skills and I loved it. That feeling was the final push that got me to this post.

I decided that I wanted to build a machine where I could update my programming and data analysis skills and start on machine learning concepts and projects. I’d built windows based desktops and Ubuntu based servers before. So building one wasn’t going to be a problem. But picking the right parts was. I read a lot of posts about people building their own deep learning/machine learning rigs for anywhere from $1,000 to $3,000. I initially aimed for $1,500 but as I started reading more I realized that I could get a good machine built for closer to $1,400. But given that I was willing to wait till thanksgiving I was able to get the price to below $1,350.

So here it is – My custom rig for less than $1,350.

CPU: AMD – Ryzen 5 1600 3.2GHz 6-Core Processor

So after the AMD FX 64 series, this seems to the first time in a long time that AMD has some compelling hardware compared to Intel. The Ryzen 5 1600 is a 6 core/12 thread processor that I bought for $175 on Amazon on Thanksgiving 2017. The processor provides more cores than the core i5 (4C/4T) at a much lower price. The other choice was a 1600x with slightly higher clock speeds but it didn’t come with  a fan . The total price including an aftermarket cooler compatible with my case would have been $240. The difference in clock speeds is negated once you realize that the 1600 can be overclocked to at or near 1600x performance numbers and at $65 less it’s the better buy.

With Intel’s Coffee Lake lineup the Core i5 now offers 6C/6T but at a price of ~ $280 at the time of purchase it was too new and still more pricier than AMD. It is however good to see Intel getting some competition again. It’s just great for consumers.

MOTHERBOARD: ASRock – AB350M Pro4 Micro ATX AM4 Motherboard

I went with this motherboard which had fairly good reviews on both Newegg,com and Amazon.com. I wanted to also keep things small, so I went with the MicroATX version of the case. I could have gone with ITX but the memory on most boards I checked was limited to 32GB. I wanted the option to expand RAM to 64GB in the future. I bought it for $75 with a $10 rebate but Newegg was kind enough to refund $15 because the price dropped within 24 hours of my purchase. Final price $49.99

CASE:  Corsair Air 240 MicroATX Mid Tower Case

This case was a pleasure to build in. Unique dual chamber layout made cable management a breeze. Got the white one. A great video review from Hardwarecanucks can be found here (https://www.youtube.com/watch?v=MgwzyBk3r6A) . Although my experience with the case finish was better than the version in this video. Maybe Corsair made changes based on feedback.

GPU:  MSI GTX 1070 AERO ITX Video card

I was expecting to spend a good amount of money on a graphics based on feedback from multiple sources. But what I found tough was coming to grips with the reality that I would be spending so much on a graphics card for something other than gaming. I have a AMD R9 290X on my gaming machine. Needless to say that this card is much faster. Tim Dettmers post is often cited as the bible but I actually found more practical advice from the builds from the following websites. Prices have changed a lot since these posts were published so make sure to look at pcpartpicker.com for pricing your build.

• https://www.servethehome.com/optimizing-a-starter-cuda-machine-learning-ai-deep-learning-build/
• https://www.oreilly.com/learning/build-a-super-fast-deep-learning-machine-for-under-1000
• https://www.servethehome.com/deeplearning02-sub-800-ai-machine-learning-cuda-starter-build/

RAM: Ballistix Sport LT 16GB Single DDR4 2666 (PC4-21300) x 2

RAM is expensive. Especially 32GB of it. Bought 2 sticks of RAM, 16GB each for a total of $295. Will upgrade to 64GB in the future if the need arises.

STORAGE:

• OCZ Vertex 2 120 GB SATA II: I had this drive lying around from an older machine. It’s not as fast as current SSDs on SATA III but it definitely is faster than HDDs. This will be the main OS drive.
• Samsung – 850 EVO-Series 500GB SATA III: One of the best SATA III drives in the market. This is the main disk for applications and databases.
• TOSHIBA N300 4TB 7200 RPM Desktop Hard Drive: When it comes to space HDDs just cannot be beat from a price/storage ratio. Got this 4TB for $110 and it comes with a 3 year warranty.  A good drive to archive any old analyses and data.

PSU:  Corsair – CSM 650W 80+ Gold Certified Semi-Modular ATX Power Supply

A good power supply is a must for any server that’s going to run serious computations. I usually go above PC Part Picker’s wattage measurements. I could have done with 550W but the 650W power supply actually had a deal and rebate, so it worked out to be cheaper.

WIFI ADAPTER: TP-LINK TL-WDN4800 Dual Band Wireless N900 PCI Express Adapter

This adapter just plain works on Ubuntu Server. I can connect to a wifi network during the setup of the OS itself. I had purchased the Edimax AC1200 Wi-Fi USB Adapter (EW-7822ULC) but had to return it because it would just not work on Linux kernels over 4.11. I had 4.13 when I installed Ubuntu Server 17.10. The TP-Link version works on both Ubuntu 16.04 and 17.10 and is more than enough for a 20Mbps internet. Why wifi and not wired? Well I didn’t want to run wires across my house and the machine will actually sit in my closet. So a wifi adapter with the wifi router across the room met my needs.