I've written a small script that will connect to a remote news server as a reader, and create a local spool from the articles. This spool matches an inn spool exactly, and will clone the remote server's spool for whatever groups you want, and however may days you configure. It is meant to be run as a cron job.

It's not done, so not releasing it yet (no error checking or tracking last article number yet).

This will allow rocksolid overboard to work on a site with no news server, and I think this may also allow bbs to run on a server without a news server. It may have uses besides these also.

Retro Guy

--
Posted on Rocksolid Light

This will allow rocksolid overboard to work on a site with no news server,

that's great. it will lower the effort to install a new node.

and I think this may also allow bbs to run on a server without a news server.

unfortunately not (or at least not if you want to do posts on the node). posting works only locally.

It may have uses besides these also.

is it a shellscript ? if nothing else, i would love to look at the code ans learn.

cheers

trw

Posted on: def3.i2p

trw wrote:

and I think this may also allow bbs to run on a server without a news

server.

unfortunately not (or at least not if you want to do posts on the node).

posting works only locally.

Does bbs use rpost? If so, I have a bash script I wrote that posts to a remote server using rpost. Seems to work fine:

rpost $newsserver -U $username -P $password <<%end
$(<message.including.headers.txt)
%end

If your outgoing message is just header and body lines of plain text, that should work. You can also send headers one by one, or whatever you want before %end.

It may have uses besides these also.

is it a shellscript ? if nothing else, i would love to look at the code ans

learn.

My spool script is written in php.

Retro Guy

--
Posted on Rocksolid Light

Currently running this script via cron to see how well it works (or doesn't :)

So far it's going well, I have a cloned rocksolid/* spool now and we'll see if it stays properly in sync.

Retro Guy

--
Posted on Rocksolid Light

Does bbs use rpost? If so, I have a bash script I wrote that posts to a remote server using rpost. Seems to work fine:

No, I have tried that before. bbs uses rnews, and the vm does not have a rpost binary.
Posted on: def3.i2p

  To: trw
trw wrote:

Does bbs use rpost? If so, I have a bash script I wrote that posts to a

remote server using rpost. Seems to work fine:

No, I have tried that before. bbs uses rnews, and the vm does not have a

rpost binary.

Ah, ok. I was thinking your were using rpost.

One thing about my script 'spoolnews.php', is that it can be modified to read articles from the server, then reformat them for injecting elsewhere. I'll try to keep the code as non-inn specific as possible.

Retro Guy


--
Posted on RetroBBS

  To: Retro Guy
Also, I just realized (why didn't I before?) that I can make a perfectly good news spool from a Synchronet installation. I'm doing that now to see how well overboard may work if the news server is other than inn.

Retro Guy

--
Posted on RetroBBS

  To: Retro Guy
Retro Guy wrote:

Also, I just realized (why didn't I before?) that I can make a perfectly good news spool from a Synchronet installation. I'm doing that now to see how well overboard may work if the news server is other than inn.

This seems to be working.

I've created and added a cron job to keep a news spool up to date on retrobbs, which has Synchronet as a news server, not inn.

The overboard for rocksolid.* and DOVE-Net.* is working, but it will take a few days to sort properly, as it uses the file timestamps to predict the article date (the date displayed is from the article).

Since the spool was created all at once, every article looks like it is new. I want to monitor over the next few days to see how it sorts out.

Retro Guy


--
Posted on RetroBBS

  To: Retro Guy
Retro Guy wrote:

The overboard for rocksolid.* and DOVE-Net.* is working, but it will take a few days to sort properly, as it uses the file timestamps to predict the article date (the date displayed is from the article).

Found a bug that saved the last article number retrieved plus one (so one article skipped each run). That's been fixed. I'll tackle bugs as they show up, which I expect plenty of at the start.

The overboard(s) on retrobbs now are running using the spoolnews script, so it doesn't matter where the news server is (Synchronet or inn, or ???).

Retro Guy
--
Posted on RetroBBS

  To: Retro Guy
Retro Guy wrote:

Found a bug that saved the last article number retrieved plus one (so one article skipped each run). That's been fixed. I'll tackle bugs as they show up, which I expect plenty of at the start.

Another bug found: Was creating zero length files for articles that don't exist on the server. Fixed.

Retro Guy

--
Posted on RetroBBS

  To: trw
Here is the 'spoolnews' php script. I'm still working on it, but this is working well so far.

You need to configure a few lines near the top (server, username etc.) and create 'groups.txt' containing just the list of newsgroups you want to spool. This fill will be modified by the script to track last saved article numbers.

Put this file, and groups.txt in the same dir, and make sure the user can write to that directory so it can create the 'articles' directory tree there. This is meant to be run as a cron job.

There are some known (to me) bugs, but it is working well.

Retro Guy

-----------------------
#!/usr/bin/php
<?php
/*  spoolnews NNTP news spool creator
 *  Version: 0.2.0
 *  Download: https://news.novabbs.com/getrslight
 *
 *  E-Mail: retroguy@novabbs.com
 *  Web: https://news.novabbs.com
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; either version 2 of the License, or
 *  (at your option) any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program; if not, write to the Free Software
 *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 */
 $server_auth_user = "<username>";
$server_auth_pass = "<password>";
$server = "news.example.com";
$port = "119";

$maxarticles = 10;
$workpath="/etc/rslight/spoolnews/";
$path=$workpath."articles/";
$groupfile="groups.txt";

# Iterate through groups $groupconfig = $workpath.$groupfile;
$grouplist = file($groupconfig, FILE_IGNORE_NEW_LINES);
foreach($grouplist as &$findgroup) {
  $name = explode(':', $findgroup);
  echo "nRetrieving articles for: ".$name[0]."...  rn";
  get_articles($name[0]);
}
echo "nDonern";

function get_articles($group) {
  global $server_auth_user, $server_auth_pass, $server, $port, $maxarticles, $workpath, $path, $groupfile;
  $grouppath = $path.preg_replace('/./', '/', $group);
    # Check if group exists. Open it if it does
  $ns=nntp_open($server, $port);
  fputs($ns, "group ".$group."rn");
  $response = line_read($ns);
  if (strcmp(substr($response,0,3),"411") == 0) {
    echo "n".$response."n";
    fclose($ns);
    return(1);
  }
    # Get config
  $groupconfig = $workpath.$groupfile;
  $grouplist = file($groupconfig, FILE_IGNORE_NEW_LINES);
  foreach($grouplist as &$findgroup) {
    $name = explode(':', $findgroup);
    if (strcmp($name[0], $group) == 0) {
      if (isset($name[1]))
        $article = $name[1] + 1;
      break;
    }
  }
    # Split group config line to get last article number
  $detail = explode(" ", $response);
  if (!isset($article)) {
    $article = $detail[2];
  }
  if($article < $detail[2]) {
    $article = $detail[2];
  }
    # Pull articles and save them in our spool
  @mkdir($grouppath,0755,'recursive');
  $i=0;
  while ($article <= $detail[3]) {
    if (file_exists($grouppath."/".$article)) {
      echo "nAlready Exists: ".$group." ".$article."n";
      $article++;
    } else {
      fputs($ns, "article ".$article."rn");
      $response = line_read($ns);
      if (strcmp(substr($response,0,3),"220") != 0) {
echo "n".$response."n";
$article++;
continue;
      }
      $response = line_read($ns);
      $articleHandle = fopen($grouppath."/".$article, 'w+');
      while(strcmp($response,".") != 0)       {
fputs($articleHandle, $response."n");
$response = line_read($ns);
      }
      @fclose($articleHandle);
      echo "nRetrieved: ".$group." ".$article."n";
      $i++;
      if($i > $maxarticles)
break;
      $article++;
    }
  }
  nntp_close($ns);
  # Save config
  $saveconfig = fopen($groupconfig, 'w+');
  foreach($grouplist as &$savegroup) {
    $name = explode(':', $savegroup);
    if (strcmp($name[0], $group) == 0) {
      fputs($saveconfig, $group.":".$article."n");
    } else {
      fputs($saveconfig, $savegroup."n");
    }
  }
  fclose($saveconfig);
}

function nntp_open($nserver=0,$nport=0) {
  global $text_error,$server_auth_user,$server_auth_pass,$readonly;
  global $server,$port;
  // echo "<br>NNTP OPEN<br>";
  $authorize=((isset($server_auth_user)) && (isset($server_auth_pass)) &&
  ($server_auth_user != ""));
  if ($nserver==0) $nserver=$server;
  if ($nport==0) $nport=$port;
  $ns=@fsockopen($nserver,$nport);
  $weg=line_read($ns);  // kill the first line
  if (substr($weg,0,2) != "20") {
    echo "<p>".$text_error["error:"].$weg."</p>";
    fclose($ns);
    $ns=false;
  } else {
    if ($ns != false) {
      fputs($ns,"MODE readerrn");
      $weg=line_read($ns);  // and once more
      if ((substr($weg,0,2) != "20") &&
((!$authorize) || ((substr($weg,0,3) != "480") && ($authorize)))) {
echo "<p>".$text_error["error:"].$weg."</p>";
      fclose($ns);
      $ns=false;
}
    }
    if ((isset($server_auth_user)) && (isset($server_auth_pass)) &&
      ($server_auth_user != "")) {
      fputs($ns,"AUTHINFO USER $server_auth_userrn");
    $weg=line_read($ns);
    fputs($ns,"AUTHINFO PASS $server_auth_passrn");
    $weg=line_read($ns);
    /* Only check auth if reading and posting same server */
    if (substr($weg,0,3) != "281" && !(isset($post_server)) && ($post_server!="")) {
      echo "<p>".$text_error["error:"]."</p>";
      echo "<p>".$text_error["auth_error"]."</p>";
    }
      }
  }
  if ($ns==false) echo "<p>".$text_error["connection_failed"]."</p>";
  return $ns;
}

/*
* Close a NNTP connection
*
* $ns: the handle of the connection
*/
function nntp_close(&$ns) {
  if ($ns != false) {
    fputs($ns,"QUITrn");
    fclose($ns);
  }
}

function line_read(&$ns) {
  if ($ns != false) {
    $t=str_replace("n","",str_replace("r","",fgets($ns,1200)));
    echo ".";
    return $t;
  }
}
?>

--
Posted on RetroBBS

  To: Retro Guy
Posting messed up the linefeeds a bit, but you can grab a copy from the 'getrslight' link on rslight.

Just rename from spoolnews.php.txt to spoolnews.php

Retro Guy

--
Posted on RetroBBS