Difference between revisions of "Usenet downloader"

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search
(New Page)
 
Line 2: Line 2:
  
 
This has been tested and works well on a ARM9 Linkstation Live. Should work on others.
 
This has been tested and works well on a ARM9 Linkstation Live. Should work on others.
Initial work:
 
You need access to the box via Freelink or Open Stock firmware or similar
 
You need IPKG installed and to have picked up the ARM9 feed from NSLU2-Linux.org
 
You need to telnet into the box as root.
 
  
Installing is made much easier by a recent update in the ARM9 feed which now includes a Hellanzb package.
+
=Prerequisites=
+
 
ipkg update
+
There are 2 options:
ipkg install py-hellanzb (or if you want to use python2.5, then ipkg install py25-hellanzb)
+
 
 +
* [[Open_Stock_Firmware]]/jtymod + [http://forum.nas-central.org/viewtopic.php?f=19&t=4075&p=39162#p39162 bootstrapping to the optware feed]
 +
* [[:Category:FreeLink]]
 +
 
 +
 
 +
 
 +
 
 +
=Stock firmware/jtymod=
 +
==Installation==
 +
 
 +
ipkg update
 +
ipkg install py-hellanzb  
 +
(or if you want to use python2.5, then ipkg install py25-hellanzb)
  
 
This installs Python, HellaNZB and all the dependencies such as Twisted, RAR, Par2Cmdline and the faster YEnc decoder  
 
This installs Python, HellaNZB and all the dependencies such as Twisted, RAR, Par2Cmdline and the faster YEnc decoder  
 
   
 
   
But you will need to configure Hellanzb
+
== Configuration ==
   
+
  cd /opt/local/bin
cd /opt/local/bin
+
vi hellanzb.conf
vi hellanzb.conf
+
 
   
 
   
 
You need to change the settings to match those of your usenet provider (username, password, server,number of connections). Remember, the Linkstation is a NAS box not a high spec PC so don't be too greedy on the number of connections as it might actually make it go slower. For example, I can happily max out a 4Mbit line with 4 connections. Try different numbers and see how you get on.
 
You need to change the settings to match those of your usenet provider (username, password, server,number of connections). Remember, the Linkstation is a NAS box not a high spec PC so don't be too greedy on the number of connections as it might actually make it go slower. For example, I can happily max out a 4Mbit line with 4 connections. Try different numbers and see how you get on.
  
# Set both the username and password to 'None' (without the quotes) if your  
+
<pre>
# usenet server does not require authorization  
+
# Set both the username and password to 'None' (without the quotes) if your  
defineServer(id = 'your usenet',  
+
# usenet server does not require authorization  
hosts = [ 'yourusenet:119' ],  
+
defineServer(id = 'your usenet',  
#hosts = [ 'news.changeme.com', 'morenews.changeme.com:8000' ],  
+
hosts = [ 'yourusenet:119' ],  
 +
#hosts = [ 'news.changeme.com', 'morenews.changeme.com:8000' ],  
  
username = 'your username',  
+
username = 'your username',  
password = 'your password',  
+
password = 'your password',  
#username = None, # no auth  
+
#username = None, # no auth  
#password = None,  
+
#password = None,  
  
connections = 4,  
+
connections = 4,  
  
# Important locations  
+
# Important locations  
Hellanzb.PREFIX_DIR = '/mnt/disk1/shared/download'  
+
Hellanzb.PREFIX_DIR = '/mnt/disk1/shared/download'  
 +
</pre>
  
 
A further important change is to fix the permissions (you want to be able to move files from the download directory presumably?) so change
 
A further important change is to fix the permissions (you want to be able to move files from the download directory presumably?) so change
Hellanzb.UMASK = 0022  to
+
<pre> Hellanzb.UMASK = 0022  </pre>
Hellanzb.UMASK = 0000
+
to
 +
<pre> Hellanzb.UMASK = 0000</pre>
  
 
Move the resulting config file to where the actual Hellanzb python script is located
 
Move the resulting config file to where the actual Hellanzb python script is located
  
mv hellanzb.conf /opt/local/bin/hellanzb.conf
+
mv hellanzb.conf /opt/local/bin/hellanzb.conf
  
 
Now there should be hellanzb.py and your hellanzb.conf there. Now you can fire up hellanzb either from command line or from script.
 
Now there should be hellanzb.py and your hellanzb.conf there. Now you can fire up hellanzb either from command line or from script.
  
cd /opt/local/bin
+
cd /opt/local/bin
python hellanzb.py -D  
+
python hellanzb.py -D  
  
 
The -D makes it run in Daemon mode, i.e it runs in the background until stopped.
 
The -D makes it run in Daemon mode, i.e it runs in the background until stopped.
Line 55: Line 65:
 
If everything went well hellanzb is running and will create the necessary sub-dirs in the dir you created  
 
If everything went well hellanzb is running and will create the necessary sub-dirs in the dir you created  
  
/mnt/disk1/shared/download
+
/mnt/disk1/shared/download
/mnt/disk1/shared/download/nzb  
+
/mnt/disk1/shared/download/nzb  
/mnt/disk1/shared/download/nzb/daemon.current  
+
/mnt/disk1/shared/download/nzb/daemon.current  
/mnt/disk1/shared/download/nzb/daemon.queue  
+
/mnt/disk1/shared/download/nzb/daemon.queue  
/mnt/disk1/shared/download/nzb/daemon.postponed  
+
/mnt/disk1/shared/download/nzb/daemon.postponed  
/mnt/disk1/shared/download/nzb/daemon.temp  
+
/mnt/disk1/shared/download/nzb/daemon.temp  
/mnt/disk1/shared/download/nzb/daemon.processing  
+
/mnt/disk1/shared/download/nzb/daemon.processing  
/mnt/disk1/shared/download/nzb/daemon.working  
+
/mnt/disk1/shared/download/nzb/daemon.working  
  
 
Just pop your NZB file in the /mnt/disk1/shared/download/nzb/daemon.queue dir and hellanzb will start downloading and pop  
 
Just pop your NZB file in the /mnt/disk1/shared/download/nzb/daemon.queue dir and hellanzb will start downloading and pop  
Line 70: Line 80:
 
python hellanzb.py status
 
python hellanzb.py status
 
   
 
   
 +
==Adding a webfrontend==
 
Realistically you will probably want a web based frontend. Hellahella doesn't seme to want to run but Zussaweb works fine with Lighttpd. You could use a 2nd instance of Apache or maybe piggyback on the Apache that runs the LS's web interface but it didn't work for me.
 
Realistically you will probably want a web based frontend. Hellahella doesn't seme to want to run but Zussaweb works fine with Lighttpd. You could use a 2nd instance of Apache or maybe piggyback on the Apache that runs the LS's web interface but it didn't work for me.
  
ipkg update
+
ipkg update
ipkg install lighttpd
+
ipkg install lighttpd fcgi-php
ipkg install fcgi-php
+
 
   
 
   
 
You'll need to configure (of course)
 
You'll need to configure (of course)
 
   
 
   
vi /opt/etc/lighttpd/lighttpd.conf
+
vi /opt/etc/lighttpd/lighttpd.conf
 
   
 
   
 
change
 
change
#                              "mod_fastcgi",
+
<pre> #                              "mod_fastcgi", </pre>
  into
+
into
                                "mod_fastcgi",
+
<pre>                                "mod_fastcgi",</pre>
change
+
change
url.access-deny            = ( "~", ".inc" )
+
<pre>url.access-deny            = ( "~", ".inc" )</pre>
  into
+
into
url.access-deny            = ( "~", ".inc", ".sqlite" )
+
<pre>url.access-deny            = ( "~", ".inc", ".sqlite" )</pre>
 
   
 
   
  change
+
change
 +
<pre>
 
#fastcgi.server            = ( ".php" =>
 
#fastcgi.server            = ( ".php" =>
 
#                              ( "localhost" =>
 
#                              ( "localhost" =>
Line 98: Line 109:
 
#                              )
 
#                              )
 
#                            )
 
#                            )
 
+
</pre>
  into
+
into
 +
<pre>
 
fastcgi.server            = ( ".php" =>
 
fastcgi.server            = ( ".php" =>
 
                               ( "localhost" =>
 
                               ( "localhost" =>
Line 108: Line 120:
 
                               )
 
                               )
 
                             )
 
                             )
+
</pre>
 
   
 
   
 
Give some thought as to where you want to have your webpages. I like mine accessible to my Windows PCs so that I can edit directly in Frontpage or whatever but in this case I won't change the defaults.. Also you may need to open a port to the webserver if you want to have your Zussaweb available remotely over the internet.
 
Give some thought as to where you want to have your webpages. I like mine accessible to my Windows PCs so that I can edit directly in Frontpage or whatever but in this case I won't change the defaults.. Also you may need to open a port to the webserver if you want to have your Zussaweb available remotely over the internet.
Line 116: Line 128:
 
Next download and unpack zussaweb:
 
Next download and unpack zussaweb:
  
cd /opt/tmp
+
cd /opt/tmp
wget http://download.sourceforge.net/sourceforge/zussaweb/zussaweb03.tar.gz
+
wget http://download.sourceforge.net/sourceforge/zussaweb/zussaweb03.tar.gz
cd /opt/share/www/lighttpd/
+
cd /opt/share/www/lighttpd/
tar -xzvf /opt/tmp/zussaweb03.tar.gz
+
tar -xzvf /opt/tmp/zussaweb03.tar.gz
  
 
Next you need to change the values of $disk and $download_nzb_path in the file /opt/share/www/lighttpd/zussaweb/settings.php
 
Next you need to change the values of $disk and $download_nzb_path in the file /opt/share/www/lighttpd/zussaweb/settings.php
Line 126: Line 138:
 
Edit /opt/share/www/lighttpd/zussaweb/functions.php and change:
 
Edit /opt/share/www/lighttpd/zussaweb/functions.php and change:
  
function freediskspace() {
+
function freediskspace() {
 
         return (round (disk_free_space(disk)/1024/1024,2));
 
         return (round (disk_free_space(disk)/1024/1024,2));
}
+
}
 
   
 
   
 
to
 
to
 
   
 
   
function freediskspace($disk) {
+
function freediskspace($disk) {
 
         return (round (disk_free_space($disk)/1024/1024,2));
 
         return (round (disk_free_space($disk)/1024/1024,2));
 
   
 
   
Line 138: Line 150:
 
Now you can use the interface at http://linkstation:8081/zussaweb/
 
Now you can use the interface at http://linkstation:8081/zussaweb/
 
   
 
   
Remaining issues needing fixing
+
==Remaining issues needing fixing==
  
1) Getting Hellanzb to start up in daemon mode when LS starts.
+
1) Getting Hellanzb to start up in daemon mode when LS starts.<br>
 
2) Getting the "upload an NZB" and "enter Newzbin ref" options in Zussaweb to work right.
 
2) Getting the "upload an NZB" and "enter Newzbin ref" options in Zussaweb to work right.
 +
 +
= Freelink =
 +
 +
==Installation==
 +
 +
==Configuration==
 +
 +
==Adding a webfrontend==

Revision as of 10:55, 28 November 2007

[Sources/Credits Methanoid & Fantasymick on NAS-central.org, Caveman on OpenFSG.com and DrChair at wl500g.info]

This has been tested and works well on a ARM9 Linkstation Live. Should work on others.

Contents

Prerequisites

There are 2 options:



Stock firmware/jtymod

Installation

ipkg update
ipkg install py-hellanzb 
(or if you want to use python2.5, then ipkg install py25-hellanzb)

This installs Python, HellaNZB and all the dependencies such as Twisted, RAR, Par2Cmdline and the faster YEnc decoder

Configuration

cd /opt/local/bin
vi hellanzb.conf

You need to change the settings to match those of your usenet provider (username, password, server,number of connections). Remember, the Linkstation is a NAS box not a high spec PC so don't be too greedy on the number of connections as it might actually make it go slower. For example, I can happily max out a 4Mbit line with 4 connections. Try different numbers and see how you get on.

 # Set both the username and password to 'None' (without the quotes) if your 
 # usenet server does not require authorization 
 defineServer(id = 'your usenet', 
 hosts = [ 'yourusenet:119' ], 
 #hosts = [ 'news.changeme.com', 'morenews.changeme.com:8000' ], 

 username = 'your username', 
 password = 'your password', 
 #username = None, # no auth 
 #password = None, 

 connections = 4, 

 # Important locations 
 Hellanzb.PREFIX_DIR = '/mnt/disk1/shared/download' 

A further important change is to fix the permissions (you want to be able to move files from the download directory presumably?) so change

 Hellanzb.UMASK = 0022   

to

 Hellanzb.UMASK = 0000

Move the resulting config file to where the actual Hellanzb python script is located

mv hellanzb.conf /opt/local/bin/hellanzb.conf

Now there should be hellanzb.py and your hellanzb.conf there. Now you can fire up hellanzb either from command line or from script.

cd /opt/local/bin
python hellanzb.py -D 

The -D makes it run in Daemon mode, i.e it runs in the background until stopped.

{edit] put script reference in here - the script is something like S71hellanzb and to run sh it. ./opt/etc/init.d/S71hellanzb If everything went well hellanzb is running and will create the necessary sub-dirs in the dir you created

/mnt/disk1/shared/download
/mnt/disk1/shared/download/nzb 
/mnt/disk1/shared/download/nzb/daemon.current 
/mnt/disk1/shared/download/nzb/daemon.queue 
/mnt/disk1/shared/download/nzb/daemon.postponed 
/mnt/disk1/shared/download/nzb/daemon.temp 
/mnt/disk1/shared/download/nzb/daemon.processing 
/mnt/disk1/shared/download/nzb/daemon.working 

Just pop your NZB file in the /mnt/disk1/shared/download/nzb/daemon.queue dir and hellanzb will start downloading and pop the unarchived resulting files in your destination directory.

You can monitor (without a web frontend) by typing python hellanzb.py status

Adding a webfrontend

Realistically you will probably want a web based frontend. Hellahella doesn't seme to want to run but Zussaweb works fine with Lighttpd. You could use a 2nd instance of Apache or maybe piggyback on the Apache that runs the LS's web interface but it didn't work for me.

ipkg update
ipkg install lighttpd fcgi-php

You'll need to configure (of course)

vi /opt/etc/lighttpd/lighttpd.conf

change

 #                               "mod_fastcgi", 

into

                                "mod_fastcgi",

change

url.access-deny             = ( "~", ".inc" )

into

url.access-deny             = ( "~", ".inc", ".sqlite" )

change

#fastcgi.server             = ( ".php" =>
#                               ( "localhost" =>
#                                 (
#                                   "socket" => "/tmp/php-fastcgi.socket",
#                                   "bin-path" => "/usr/local/bin/php"
#                                 )
#                               )
#                             )

into

fastcgi.server             = ( ".php" =>
                               ( "localhost" =>
                                 (
                                   "socket" => "/tmp/php-fastcgi.socket",
                                   "bin-path" => "/opt/bin/php-fcgi"
                                 )
                               )
                            )

Give some thought as to where you want to have your webpages. I like mine accessible to my Windows PCs so that I can edit directly in Frontpage or whatever but in this case I won't change the defaults.. Also you may need to open a port to the webserver if you want to have your Zussaweb available remotely over the internet.

Now you are ready to start lighttpd by /opt/etc/init.d/S80lighttpd

Next download and unpack zussaweb:

cd /opt/tmp
wget http://download.sourceforge.net/sourceforge/zussaweb/zussaweb03.tar.gz
cd /opt/share/www/lighttpd/
tar -xzvf /opt/tmp/zussaweb03.tar.gz

Next you need to change the values of $disk and $download_nzb_path in the file /opt/share/www/lighttpd/zussaweb/settings.php

Finally we need to fix a bug: Edit /opt/share/www/lighttpd/zussaweb/functions.php and change:

function freediskspace() {
       return (round (disk_free_space(disk)/1024/1024,2));
}

to

function freediskspace($disk) {
       return (round (disk_free_space($disk)/1024/1024,2));


Now you can use the interface at http://linkstation:8081/zussaweb/

Remaining issues needing fixing

1) Getting Hellanzb to start up in daemon mode when LS starts.
2) Getting the "upload an NZB" and "enter Newzbin ref" options in Zussaweb to work right.

Freelink

Installation

Configuration

Adding a webfrontend