Posted on June 22, 2017 at 23:40 by GreenFrog
Linux Download Server (LDS) is an automation system that will keeps track of all your TV Shows, Movies and Music.
System is based on Ubuntu Server 16.04.2 LTS connected to Internet via VPN for security. Server incorporate Deluge as Torrent Client, Sonarr as Content Manager (TV Shows), Radarr Content Manager (Movies), Jackett Torrent Socket and Headphones Content Manager (Music)
Below article describes deployment process on Ubuntu Server 16.04.2 LTS but it will be more the same on more modern version.
Info There is one decision to make before deploying anything. You can deploy Ubuntu Server as standalone server on bear metal - desktop/laptop etc. or you might consider having it installed as virtual machine using i.e. Virtual Box or VMware Workstation.
I would encourage you to use any virtualization platform for many reasons. By having server as a VM you will be able to snapshot your system at any time and go back to that snapshot if something will go wrong, you can also migrate your system to another server hardware if needed in the future and not having to go thought all of the configuration again.
I personally use QNAP NAS that offer its own virtualization platform. I use it mainly because my QNAP NAS is always ON and is hidden from site where only noise I get is from my laptop and not from array of disks, trust me you will need substantial disk capacity if system that you are about to build will work as it should.
I won’t be telling you how to deploy Linux OS - just install it somewhere, depending on how that installation is done you might not need SSH access or VPN. If you install Ubuntu on a remote hardware you will obviously need some SSH access into it and VPN might be necessary if i.e. Torrent traffic is blocked by your ISP.
Just because my Ubuntu server is remote I need to install SSH service on it first so I can manage it remotely.
Install SSH server:
sudo apt-get install openssh-server
After installation is completed we should be able to connect to the server via PuTTY
sudo apt install openvpn
Now, if you choose a VPN provider (paid or free) they should provide you with *.ovpn file - this file contain connection details that you should use while connecting to the VPN provider servers.
If you decided which VPN server you are going to use, select relevant *.ovpn file provided and copy it to your Ubuntu server - copy it in to your home folder for now.
*.ovpn file name to
sudo mv current_name.ovpn client.conf
client.conf file in to
sudo cp -r client.conf /etc/openvpn/client.conf
Edit copied file and find section called
auth-user-pass and change it to
auth-user-pass auth.txt save and exit Ctrl + O and Ctrl + X
sudo nano /etc/openvpn/client.conf
sudo nano /etc/openvpn/auth.txt
Put your VPN username and password in to
auth.txt file, save and exit.
Now we need to make sure that OpenVPN client will start each time server is rebooted.
sudo nano /etc/default/openvpn
#AUTOSTART="all" and remove "#" then save and exit Ctrl + O and Ctrl + X
Reload OpenVPN daemon:
sudo systemctl daemon-reload
OpenVPN is a templated service under systemd. The service is named
[email protected]. So you should be starting your VPN service using file you copied earlier
/etc/openvpn/client.conf where .conf file name is incorporated in to service enable and start command as below:
systemctl enable [email protected] systemctl start [email protected] systemctl status [email protected]
You should see something like below after typing commands from above:
BTW. my config.conf file is named nordvpn.conf therefore your output will look slightly different.
● [email protected] - OpenVPN connection to nordvpn Loaded: loaded (/lib/systemd/system/[email protected]; disabled; vendor preset: enabled) Active: active (running) since Sun 2017-07-09 12:39:15 IST; 1min 6s ago Docs: man:openvpn(8) https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage https://community.openvpn.net/openvpn/wiki/HOWTO Process: 1765 ExecStart=/usr/sbin/openvpn --daemon ovpn-%i --status /run/openvpn/%i.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/%i.conf --writepid /run/ Main PID: 1769 (openvpn) CGroup: /system.slice/system-openvpn.slice/[email protected] └─1769 /usr/sbin/openvpn --daemon ovpn-nordvpn --status /run/openvpn/nordvpn.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/nordvpn.conf --writepi Jul 09 12:39:18 lds ovpn-nordvpn: ROUTE_GATEWAY 192.168.88.1/255.255.255.0 IFACE=ens3 HWADDR=00:54:00:01:b6:0f Jul 09 12:39:18 lds ovpn-nordvpn: TUN/TAP device tun0 opened Jul 09 12:39:18 lds ovpn-nordvpn: TUN/TAP TX queue length set to 100 Jul 09 12:39:18 lds ovpn-nordvpn: do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0 Jul 09 12:39:18 lds ovpn-nordvpn: /sbin/ip link set dev tun0 up mtu 1500 Jul 09 12:39:18 lds ovpn-nordvpn: /sbin/ip addr add dev tun0 10.56.65.12/24 broadcast 10.56.65.255 Jul 09 12:39:18 lds ovpn-nordvpn: /sbin/ip route add 126.96.36.199/32 via 192.168.88.1 Jul 09 12:39:18 lds ovpn-nordvpn: /sbin/ip route add 0.0.0.0/1 via 10.56.65.1 Jul 09 12:39:18 lds ovpn-nordvpn: /sbin/ip route add 188.8.131.52/1 via 10.56.65.1 Jul 09 12:39:18 lds ovpn-nordvpn: Initialization Sequence Completed
Now when typing
ifconfig you should see additional network interface named
tun0 and IP address that is obtained from your VPN provider.
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.56.65.12 P-t-P:10.56.65.12 Mask:255.255.255.0 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:19 errors:0 dropped:0 overruns:0 frame:0 TX packets:33 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:6797 (6.7 KB) TX bytes:4936 (4.9 KB)
At this point you can reboot your server to make sure that after is server is back you still see tun0 interface, if you do, your Internet connection is secure and entire traffic from the server that goes outside your LAN is pushed through encrypted connection to your VPN provider server.
Jackett is another service that act as a middleman between Torrent trackers and Sonarr or CouchPotato, it allows them to find relevant torrent files and puts them in to Torrent client for download - without that nothing will be downloaded automatically.
sudo apt-get install libcurl4-openssl-dev sudo apt-get install mono-devel
Go in to https://github.com/Jackett/Jackett/releases/ and check latest release of Jackett, then replace below link with code version that is most recent on GitHub, in my case at present it is v0.7.1533. Go in to /root/ directory, download and unzip Jackett.Binaries.Mono.tar.gz as shown below.
cd /root/ wget -q https://github.com/Jackett/Jackett/releases/download/v0.7.1533/Jackett.Binaries.Mono.tar.gz tar zxvf Jackett.Binaries.Mono.tar.gz
Make service out of it:
cat > jackett.service << EOF [Unit] Description=Jackett Daemon After=network.target [Service] WorkingDirectory=/root/Jackett/ User=root Restart=always RestartSec=5 Type=simple ExecStart=/usr/bin/mono /root/Jackett/JackettConsole.exe --NoRestart TimeoutStopSec=20 [Install] WantedBy=multi-user.target EOF
Enable and start Jackett:
sudo mv jackett.service /lib/systemd/system/jackett.service sudo systemctl enable jackett sudo systemctl start jackett sudo systemctl status jackett sudo apt-get update
Open http://localhost:9117 - you might see a warning message at the bottom of the page
To check mono version type
mono --version, we should get it updated from 4.2 to 5.0 and Jacket should no longer display warning message.
[email protected]:~$ mono --version Mono JIT compiler version 4.2.1 (Debian 184.108.40.206+dfsg2-7ubuntu4) Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
Update Mono to 5.0:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF echo "deb http://download.mono-project.com/repo/ubuntu xenial main" | sudo tee /etc/apt/sources.list.d/mono-official.list sudo apt-get install mono-complete sudo apt-get update sudo apt-get upgrade
Check Mono version again:
[email protected]:~$ mono --version Mono JIT compiler version 220.127.116.11 (2017-02/5077205 Thu May 25 09:19:18 UTC 2017) Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
Reload Jackett page, if Jackett still display warning message just restart jackett service
sudo systemctl stop jackett sudo systemctl start jackett
Go to the Jackett configuration section at the bottom of the page or just continue installing remaining pieces reaching configuration section when all is installed first.
Deluge is a torrent client that I decided to use in this deployment. It will communicate with Sonarr, CouchPotato and Headphones receiving request for download and when download is completed these programs will move relevant files in to relevant folders on your disk. Downloaded files will be renamed and put in to appropriate folder structure, so you have all files properly named and located in relevant places not all over the place.
sudo touch /var/log/deluged.log sudo touch /var/log/deluge-web.log sudo chown root:root /var/log/deluge* sudo apt install deluged sudo apt install deluge-webui sudo apt install deluge-console
Create Deluge service:
cat > deluged.service << EOF [Unit] Description=Deluge Bittorrent Client Daemon After=network-online.target [Service] Type=simple User=root Group=root UMask=077 ExecStart=/usr/bin/deluged -d Restart=on-failure # Configures the time to wait before service is stopped forcefully. TimeoutStopSec=300 [Install] WantedBy=multi-user.target EOF
Enable and start service.
sudo mv deluged.service /lib/systemd/system/deluged.service sudo systemctl enable deluged sudo systemctl start deluged sudo systemctl status deluged
Install Deluge web interface:
cat > deluged-web.service << EOF [Unit] Description=Deluge Bittorrent Client Web Interface After=network-online.target [Service] Type=simple User=root Group=root UMask=027 ExecStart=/usr/bin/deluge-web Restart=on-failure [Install] WantedBy=multi-user.target EOF
Enable and start web UI
sudo mv deluged-web.service /lib/systemd/system/deluged-web.service sudo systemctl enable deluged-websudo systemctl start deluged-web sudo systemctl status deluged-web
From any machine try to get to your Deluge web UI http://localhost:8112 if you not seeing any website maybe it is Ubuntu firewall that is blocking it - disable it while testing
sudo ufw disable
Try again - if you see Password prompt put deluge as default password.
Go to the Deluge configuration section at the bottom of the page or just continue installing remaining pieces reaching configuration section when all is installed first.
Sonarr.tv - is automation system that organize and tracks all what should be downloaded and does that when new episodes are available - it search torrent trackers and puts found torrents in to torrent client i.e. Deluge. When download is completed Sonnat changes file name and puts it in specific place on a disk. https://sonarr.tv/
Installation - https://github.com/Sonarr/Sonarr/wiki/Installation
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys FDA5DFFC echo "deb http://apt.sonarr.tv/ master main" | sudo tee /etc/apt/sources.list.d/sonarr.list sudo apt-get install nzbdrone
Test that you can start Sonar manually:
mono --debug /opt/NzbDrone/NzbDrone.exe
You can also try to access web UI http://localhost:8989 to see if Sonarr web UI is available.
Let’s make Sonarr start automatically after reboot:
cat > sonarr.service << EOF [Unit] Description=Sonarr Daemon After=syslog.target network.target [Service] User=root Group=root Type=simple ExecStart=/usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser TimeoutStopSec=20 KillMode=process Restart=on-failure [Install] WantedBy=multi-user.target EOF
Create Sonarr service:
sudo mv sonarr.service /lib/systemd/system/sonarr.service systemctl enable sonarr.service systemctl start sonarr.service systemctl status sonarr.service
Later on when you will be trying to add any torrent indexer in Sonarr from Jackett you might get an error regarding SSL certificates, so to avoid it we can do some adjustment now so you don’t have to wonder what happened later.
Error message when testing indexers in Sonarr:
sudo apt-get install ca-certificates sudo apt-get install openssl sudo cat /etc/ssl/certs/*.pem > /tmp/ca-certs.pem sudo cert-sync /tmp/ca-certs.pem sudo rm /tmp/ca-certs.pem
Go to the Sonarr configuration section at the bottom of the page or just continue installing remaining pieces reaching configuration section when all is installed first.
Radarr is a fork of Sonarr with the difference that this one is for movies not for serials, This development is claiming space of CouchPotato which is used for that exact purpose. So you can choose to use Radarr or CouchPotato but you can install both if you wish and use one of them.
Install Radarr https://github.com/Radarr/Radarr/wiki/Installation If link does not work search GitHub for Radarr
Download and extract the latest release from github, changing the url below to match.
cd /root/ wget -q https://github.com/Radarr/Radarr/releases/download/v0.2.0.870/Radarr.develop.0.2.0.870.linux.tar.gz tar zxvf Radarr.develop.0.2.0.870.linux.tar.gz -C /opt/
Create Radarr service:
cat > radarr.service << EOF [Unit] Description=Radarr Daemon After=syslog.target network.target [Service] User=root Group=root Restart=always RestartSec=5 Type=simple ExecStart=/usr/bin/mono --debug /opt/Radarr/Radarr.exe --nobrowser TimeoutStopSec=20 KillMode=process [Install] WantedBy=multi-user.target EOF
Start Radar Service
sudo mv radarr.service /lib/systemd/system/radarr.service systemctl enable radarr.service systemctl start radarr.service systemctl status radarr.service
Radarr is now installed as a service set to auto start, check that it is running by opening http://localhost:7878 in your browser. If the web-ui opens and works, installation is complete.
Install headphones https://github.com/rembo10/headphones
cd ~ git clone https://github.com/rembo10/headphones.git cd ~
Create a configuration file
cat > ~/.headphones << EOF [General] http_port = 1234 # 0.0.0.0 allows outside connections. Use 'localhost' to restrict http_host = 0.0.0.0 EOF
Create Headphones service:
cat > headphones.service << EOF [Unit] Description=Headphones Daemon After=network.target [Service] User=root Restart=always RestartSec=5 Type=simple ExecStart=/usr/bin/python /root/headphones/Headphones.py --config /root/.headphones TimeoutStopSec=20 [Install] WantedBy=multi-user.target EOF
Enable and start it
sudo mv headphones.service /lib/systemd/system/headphones.service sudo systemctl enable headphones sudo systemctl start headphones sudo systemctl status headphones
If you like me, keeping your storage separate from computing by i.e. having some network shared storage I am sure you would like this system to move completed downloads to appropriate folder structure on NAS. In order for LDS to copy downloaded files in to a shared volume, from where i.e. Kodi will be pulling data for streaming, you have to map shared volume to LDS like shown below.
Install SMB utilities:
sudo apt-get install cifs-utils
Create folders that system will use for mappings:
sudo mkdir /mnt/Serials sudo mkdir /mnt/Music
If your share require authentication, create file that will hold username and password this is required to access network share:
sudo nano ~/smbcredentials
Enter your Windows username and password in the file:
Change the permissions of the file to prevent unwanted access to your credentials:
chmod 600 ~/smbcredentials
Then edit your /etc/fstab file (with root privileges) to add lines:
//192.168.11.77/Multimedia /mnt/Multimedia cifs iocharset=utf8,credentials=/root/smbcredentials,uid=1000 0 0 //192.168.11.45/DownloadsTmp /mnt/DownloadsTmp cifs iocharset=utf8,credentials=/root/smbcredentials,uid=1000 0 0
Save the file, exit the editor.
Finally, test the fstab entry by issuing:
sudo mount -a
If there are no errors, you should test how it works after a reboot. Your remote share should mount automatically.
Open http://localhost:9117 - Replace
localhost with IP address of the server where you installed Jackett.
After successful connection to Jackett web page, You should get something like shown on the picture below, currently with no indexers configured, and all what you need to do is to add couple of indexer records that we will use later on with Sonarr, Radarr and Headphones to search for available torrent files.
Find Torrent provider you like and click
+ to add indexer to Jackett.
Bear in mind that most of the indexers require account and in order to use them you need to be a member on indexer web site.
Select Torrent provider that has few Categories available therefore you can use just one for i.e. Serials, Movies and Music.
Green padlock under Type usually mean you can use torrent provider with no additional registration, but check to be sure.
If you cannot find your usual Torrent provider it means it has no API that work with Jackett
and cannot be added. For test use one of the unlocked providers, later on you can add more if needed.
If selected Torrent provider require an account you will see prompt to put your credentials while added it to Jackett.
When you successfully add one or more indexers test them, all together or each one separately, you want to see green tick against indexer you selected - what means that API is working properly and you should be able to use it with i.e. Sonarr later on.
When your Jackett configuration looks similar to the one shown on first picture above you are ready to go, keep Jackett web page open as you will need some information from it later on.
Open Deluge web UI http://localhost:8112
localhost with IP address of the server where you installed Deluge.
Log in using default password deluge.
Check if you are connected to deluge daemon, if online/offline just select and click Connect. Deluge should connect to Web UI service by default but sometimes it does not work, usually after server reboot.
Below few settings that I modify to match my preferences pushing all downloads to remote storage (SAMBA shared CIFs), you can store all locally but in my case this server is only to operate as a node for downloading where all the downloaded resources are kept remotely so nothing is lost when server is down, paused or power off - data are still available to home entertainment system like Kodi.
Bandwidth settings - limit them as you like globally, I also use a plugging that limits these values even more. There are different values during the day are different during nigh time - see last picture in this section.
Daemon section - leve port as default and enable remote connection it will be needed later.
Plugins - I use only two as you see below.
Scheduler - allows you to limit bandwidth depending on the time of the day - it overrides global settings if set higher.
Access Sonarr Web UI http://localhost:8989
localhost with IP address of the server where you installed Sonarr.
Navigate to Settings / Indexers (you should have no Indexers configured yet), Click on the big
Under Torrent section select
Torznab - Custom
Now we need some information from Jackett that we configured earlier.
First give Sonarr Indexer a name (I use the same name that I configured Jackett indexer with).
In URL field paste link copied from Jackett as shown below.
In API Key paste string value from Jackett API Key as shown below.
After setting a Name, URL and API Key leave all remaining fields as default and hit Test button at the bottom, after few seconds you should see at the bottom of the screen Sonarr reporting that it can communicate with Indexer what essentially means that both programs work together now.
Now let's add Download Client - in our case Deluge that we installed earlier.
Settings - Download Client and again press big
Under Torrent section select
At this point only what is needed is a Name (I called it Deluge).
Leave all the other settings default and hit that Test button to check if Sonarr can comunicate with Deluge.
As you see there is also an option to use Usenet network instead of Torrent one but I wont be explaining what is it and how to use it, I am sure you will be able to find plenty information on that subject elsewhere.
I wont be explaining how to configure Raddar as this is exactly the same as above Sonarr configuration, to be hones all look the same.
Headphones configuration is little more time consuming and I will leave it for another time.
I hope above will be enough for you to discover what this system does and what you can do with it yourself.
With your support anything is possible