Rocksolid Light

Welcome to novaBBS

register   nodelist   faq  

Your account also provides newsreader access to all groups at news.i2pn2.org port 119 or 563 (TLS)


rocksolid / Linux / Re: simple newsspool creator

SubjectAuthor
* simple newsspool creatorRetro Guy
`* Re: simple newsspool creatortrw
 +* Re: simple newsspool creatorRetro Guy
 |+- Re: simple newsspool creatorRetro Guy
 |`* Re: simple newsspool creatortrw
 | `* Re: simple newsspool creatorRetro Guy
 |  `* Re: simple newsspool creatorRetro Guy
 |   `* Re: simple newsspool creatorRetro Guy
 |    `* Re: simple newsspool creatorRetro Guy
 |     `- Re: simple newsspool creatorRetro Guy
 `* Re: simple newsspool creatorRetro Guy
  `- Re: simple newsspool creatorRetro Guy

Pages:12
Subject: Re: simple newsspool creator
From: retro....@retrobbs.rocksolidbbs.com.remove-9v2-this (Retro Guy)
Newsgroups: rocksolid.shared.linux
Organization: RetroBBS
Date: Sun, 22 Sep 2019 02:34 UTC
Path: i2pn2.org!rocksolid2!.POSTED.rocksolid3!not-for-mail
From: retro....@retrobbs.rocksolidbbs.com.remove-9v2-this (Retro Guy)
Newsgroups: rocksolid.shared.linux
Subject: Re: simple newsspool creator
Date: Sun, 22 Sep 2019 02:34:41 +0000
Organization: RetroBBS
Message-ID: <ec52c0c8c304330c4cf219466d6e49c3$1@www.rocksolidbbs.com>
References: <qltp0d$j87$1@def2.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: novabbs.com; posting-account="rslight.i2p"; posting-host="rocksolid3:192.241.178.238";
logging-data="22541"; mail-complaints-to="usenet@novabbs.com"
User-Agent: rslight (http://news.novabbs.com)
To: trw
X-Comment-To: trw
In-Reply-To: <qltp0d$j87$1@def2.org>
X-FTN-PID: Synchronet 3.17a-Linux Dec 29 2018 GCC 6.3.0
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on rocksolidbbs.com
X-Rslight-Site: $2y$10$tR2LGXqEINYcAY9V8M1wA.fLKiELIxVA8mma6bpRlqhvY23Zaopty
X-Gateway: retrobbs.rocksolidbbs.com [Synchronet 3.17a-Linux NewsLink 1.110]
View all headers
  To: trw
Here is the 'spoolnews' php script. I'm still working on it, but this is working well so far.

You need to configure a few lines near the top (server, username etc.) and create 'groups.txt' containing just the list of newsgroups you want to spool. This fill will be modified by the script to track last saved article numbers.

Put this file, and groups.txt in the same dir, and make sure the user can write to that directory so it can create the 'articles' directory tree there. This is meant to be run as a cron job.

There are some known (to me) bugs, but it is working well.

Retro Guy

-----------------------
#!/usr/bin/php
<?php
/*  spoolnews NNTP news spool creator
 *  Version: 0.2.0
 *  Download: https://news.novabbs.com/getrslight
 *
 *  E-Mail: retroguy@novabbs.com
 *  Web: https://news.novabbs.com
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; either version 2 of the License, or
 *  (at your option) any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program; if not, write to the Free Software
 *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 */
 $server_auth_user = "<username>";
$server_auth_pass = "<password>";
$server = "news.example.com";
$port = "119";

$maxarticles = 10;
$workpath="/etc/rslight/spoolnews/";
$path=$workpath."articles/";
$groupfile="groups.txt";

# Iterate through groups $groupconfig = $workpath.$groupfile;
$grouplist = file($groupconfig, FILE_IGNORE_NEW_LINES);
foreach($grouplist as &$findgroup) {
  $name = explode(':', $findgroup);
  echo "nRetrieving articles for: ".$name[0]."...  rn";
  get_articles($name[0]);
}
echo "nDonern";

function get_articles($group) {
  global $server_auth_user, $server_auth_pass, $server, $port, $maxarticles, $workpath, $path, $groupfile;
  $grouppath = $path.preg_replace('/./', '/', $group);
    # Check if group exists. Open it if it does
  $ns=nntp_open($server, $port);
  fputs($ns, "group ".$group."rn");
  $response = line_read($ns);
  if (strcmp(substr($response,0,3),"411") == 0) {
    echo "n".$response."n";
    fclose($ns);
    return(1);
  }
    # Get config
  $groupconfig = $workpath.$groupfile;
  $grouplist = file($groupconfig, FILE_IGNORE_NEW_LINES);
  foreach($grouplist as &$findgroup) {
    $name = explode(':', $findgroup);
    if (strcmp($name[0], $group) == 0) {
      if (isset($name[1]))
        $article = $name[1] + 1;
      break;
    }
  }
    # Split group config line to get last article number
  $detail = explode(" ", $response);
  if (!isset($article)) {
    $article = $detail[2];
  }
  if($article < $detail[2]) {
    $article = $detail[2];
  }
    # Pull articles and save them in our spool
  @mkdir($grouppath,0755,'recursive');
  $i=0;
  while ($article <= $detail[3]) {
    if (file_exists($grouppath."/".$article)) {
      echo "nAlready Exists: ".$group." ".$article."n";
      $article++;
    } else {
      fputs($ns, "article ".$article."rn");
      $response = line_read($ns);
      if (strcmp(substr($response,0,3),"220") != 0) {
echo "n".$response."n";
$article++;
continue;
      }
      $response = line_read($ns);
      $articleHandle = fopen($grouppath."/".$article, 'w+');
      while(strcmp($response,".") != 0)       {
fputs($articleHandle, $response."n");
$response = line_read($ns);
      }
      @fclose($articleHandle);
      echo "nRetrieved: ".$group." ".$article."n";
      $i++;
      if($i > $maxarticles)
break;
      $article++;
    }
  }
  nntp_close($ns);
  # Save config
  $saveconfig = fopen($groupconfig, 'w+');
  foreach($grouplist as &$savegroup) {
    $name = explode(':', $savegroup);
    if (strcmp($name[0], $group) == 0) {
      fputs($saveconfig, $group.":".$article."n");
    } else {
      fputs($saveconfig, $savegroup."n");
    }
  }
  fclose($saveconfig);
}

function nntp_open($nserver=0,$nport=0) {
  global $text_error,$server_auth_user,$server_auth_pass,$readonly;
  global $server,$port;
  // echo "<br>NNTP OPEN<br>";
  $authorize=((isset($server_auth_user)) && (isset($server_auth_pass)) &&
  ($server_auth_user != ""));
  if ($nserver==0) $nserver=$server;
  if ($nport==0) $nport=$port;
  $ns=@fsockopen($nserver,$nport);
  $weg=line_read($ns);  // kill the first line
  if (substr($weg,0,2) != "20") {
    echo "<p>".$text_error["error:"].$weg."</p>";
    fclose($ns);
    $ns=false;
  } else {
    if ($ns != false) {
      fputs($ns,"MODE readerrn");
      $weg=line_read($ns);  // and once more
      if ((substr($weg,0,2) != "20") &&
((!$authorize) || ((substr($weg,0,3) != "480") && ($authorize)))) {
echo "<p>".$text_error["error:"].$weg."</p>";
      fclose($ns);
      $ns=false;
}
    }
    if ((isset($server_auth_user)) && (isset($server_auth_pass)) &&
      ($server_auth_user != "")) {
      fputs($ns,"AUTHINFO USER $server_auth_userrn");
    $weg=line_read($ns);
    fputs($ns,"AUTHINFO PASS $server_auth_passrn");
    $weg=line_read($ns);
    /* Only check auth if reading and posting same server */
    if (substr($weg,0,3) != "281" && !(isset($post_server)) && ($post_server!="")) {
      echo "<p>".$text_error["error:"]."</p>";
      echo "<p>".$text_error["auth_error"]."</p>";
    }
      }
  }
  if ($ns==false) echo "<p>".$text_error["connection_failed"]."</p>";
  return $ns;
}

/*
* Close a NNTP connection
*
* $ns: the handle of the connection
*/
function nntp_close(&$ns) {
  if ($ns != false) {
    fputs($ns,"QUITrn");
    fclose($ns);
  }
}

function line_read(&$ns) {
  if ($ns != false) {
    $t=str_replace("n","",str_replace("r","",fgets($ns,1200)));
    echo ".";
    return $t;
  }
}
?>

--
Posted on RetroBBS



Subject: Re: simple newsspool creator
From: retro....@retrobbs.rocksolidbbs.com.remove-r5e-this (Retro Guy)
Newsgroups: rocksolid.shared.linux
Organization: RetroBBS
Date: Sun, 22 Sep 2019 02:42 UTC
Path: i2pn2.org!rocksolid2!.POSTED.rocksolid3!not-for-mail
From: retro....@retrobbs.rocksolidbbs.com.remove-r5e-this (Retro Guy)
Newsgroups: rocksolid.shared.linux
Subject: Re: simple newsspool creator
Date: Sun, 22 Sep 2019 02:42:41 +0000
Organization: RetroBBS
Message-ID: <e3a8da6e5153d2434636317238627d67$1@www.rocksolidbbs.com>
References: <ec52c0c8c304330c4cf219466d6e49c3$1@www.rocksolidbbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: novabbs.com; posting-account="rslight.i2p"; posting-host="rocksolid3:192.241.178.238";
logging-data="25631"; mail-complaints-to="usenet@novabbs.com"
User-Agent: rslight (http://news.novabbs.com)
To: Retro Guy
X-Comment-To: Retro Guy
In-Reply-To: <ec52c0c8c304330c4cf219466d6e49c3$1@www.rocksolidbbs.com>
X-FTN-PID: Synchronet 3.17a-Linux Dec 29 2018 GCC 6.3.0
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on rocksolidbbs.com
X-Rslight-Site: $2y$10$0T19WXzFuZmLD75KHgnvnesFE2cN7E84XqLF6TYm.l0IA0Cxqk3uy
X-Gateway: retrobbs.rocksolidbbs.com [Synchronet 3.17a-Linux NewsLink 1.110]
View all headers
  To: Retro Guy
Posting messed up the linefeeds a bit, but you can grab a copy from the 'getrslight' link on rslight.

Just rename from spoolnews.php.txt to spoolnews.php

Retro Guy

--
Posted on RetroBBS



Pages:12
rocksolid light 0.6.6
clearnet i2p tor