The Newsbeuter RSS Feedreader

Andreas Krennmair


Table of Contents

Introduction
Why "Newsbeuter"?
Installation
Downloading Newsbeuter
Dependencies
Compiling and Installing
First Steps
Example Configuration
Configuring Colors
Advanced Features
Tagging
Scripts and Filters (Snownews Extensions)
Podcast Support
Feedback
License

Introduction

Newsbeuter is an RSS feedreader. RSS is a number of widely-used XML formats to transmit, publish and syndicate articles, for example news or blog articles. Newsbeuter is designed to be used on text terminals on Unix or Unix-like systems such as Linux, BSD or Mac OS X.

Why "Newsbeuter"?

"Newsbeuter" is a pun on the German word "Wildbeuter", which means "hunter-gatherer". During the stone age, people hunted and gathered their food, and these days, they hunt and gather news and information. Credits for this idea goes to Clifford Wolf, who submitted it to a little competiton that was started when I got aware that the original name would violate French and European registered trademarks.

Installation

This chapter describes how to compile and install newsbeuter from source.

Downloading Newsbeuter

Newsbeuter is available as source package. Simply go to http://synflood.at/newsbeuter.html and download the latest source package, which is usually in the .tar.gz file format. Alternatively, you can check out the latest development source tree from the newsbeuter SVN repository by running the following command on the commandline:

svn co http://bereshit.synflood.at/svn/noos/trunk

Dependencies

Newsbeuter depends on a number of libraries to function correctly. This table lists these dependencies. Please be aware that the list libraries may themselves depend on other libraries. These dependencies are not listed here. Please also be aware that you need a recent C++ compiler. Currently, newsbeuter has only been tested with GCC.

Compiling and Installing

After you've downloaded and installed the dependencies mentioned above, you can start compiling and installing newsbeuter. To compile newsbeuter, simply run "make" in the source tree. After a short time, this should complete successfully, and you can go on with installation by running "make install". By default, this will install the "newsbeuter" binary to the /usr/local/bin directory. You can provide an alternative installation path using the prefix parameter, e.g. running "make install prefix=/opt/newsbeuter" will install the binary to the directory /opt/newsbeuter/bin.

First Steps

After you've installed newsbeuter, you can run it for the first time by typing "newsbeuter" on your command prompt. This will bring you the following message:

Error: no URLs configured. Please fill the file /Users/ak/.newsbeuter/urls with RSS feed URLs or import an OPML file.
newsbeuter 0.4
usage: ./newsbeuter [-i <file>|-e] [-u <urlfile>] [-c <cachefile>] [-h]
        -r              refresh feeds on start
        -e              export OPML feed to stdout
        -i <file>       import OPML file
        -u <urlfile>    read RSS feed URLs from <urlfile>
        -c <cachefile>  use <cachefile> as cache file
        -C <configfile> read configuration from <configfile>
        -v              clean up cache thoroughly
        -h              this help

This means that newsbeuter can't start without any configured feeds. To add feeds to newsbeuter, you can either add URLs to the configuration file $HOME/.newsbeuter/urls or you can import an OPML file by running "newsbeuter -i blogroll.opml". To manually add URLs, open the file with your favorite text editor and add the URLs, one per line:

http://rss.cnn.com/rss/cnn_topstories.rss
http://newsrss.bbc.co.uk/rss/newsonline_world_edition/front_page/rss.xml

If you need to add URLs that have restricted access via username/password, simply provide the username/password in the following way:

http://username:password@hostname.domain.tld/feed.rss

In order to protect username and password, make sure that $HOME/.newsbeuter/urls has the appropriate permissions.

Now you can run newsbeuter again, and it will present you with a controllable list of the URLs that you configured previously. You can now start downloading the feeds, either by pressing "R" to download all feeds, or by pressing "r" to download the currently selected feed. You can then select a feed you want to read, and by pressing "Enter", you can go to the article list for this feed. This works even while the downloading is still in progress. You can now see the list of available articles by their title. A "N" on the left indicates that an article wasn't read yet. Pressing Enter brings you to the content of the article. You can scroll through this text, and also run a browser (default: lynx) to view the complete article if the content is empty or just an abstract or a short description. Pressing "q" brings you back to the article list, and pressing "q" again brings you back to the feed list. Pressing "q" a third time then closes newsbeuter.

Newsbeuter caches the article that it downloads. This means that when you start newsbeuter again and reload a feed, the old articles can still be read even if they aren't in the current RSS feeds anymore. Optionally you can configure how many articles shall be preserved by feed so that the article backlog doesn't grow endlessly (see below).

Several aspects of newsbeuter's behaviour can be configured via a configuration file, by default $HOME/.newsbeuter/config. This configuration files contains lines in the form "<config-command> <arg1> …". The configuration file can also contain comments, which start with the # character and go as far as the end of line. If you need to enter a configuration argument that contains spaces, use quotes (") around the whole argument.

Table 1. Configuration Commands

Configuration Command Argument(s) Default Description Example
show-read-feeds [yes/no] yes If yes, then all feeds, including those without unread articles, are listed. If no, then only feeds with one or more unread articles are list. show-read-feeds no
browser <browser-command> lynx Set the browser command to use when opening an article in the browser. browser w3m
max-items <number> 0 Set the number of articles to maximally keep per feed. If the number is set to 0, then all articles are kept. max-items 100
bind-key <key> <operation> Bind key <key> to <operation>. This means that whenever <key> is preseed, then <operation> is executed (if applicable in the current context). A list of available operations can be found below. bind-key ^r reload-all
unbind-key <key> Unbind key <key>. This means that no operation is called when <key> is pressed. unbind-key R
use-proxy [yes/no] no If yes, then the configured proxy will be used for downloading the RSS feeds. use-proxy yes
proxy <server:port> Set the proxy to use for downloading RSS feeds. proxy localhost:3128
proxy-auth <auth> Set the proxy authentication string. proxy-auth user:password
color <element> <fgcolor> <bgcolor> [<attr> …] Set the foreground color, background color and optional attributes for a certain element color background white black
auto-reload [yes/no] no If enabled, all feeds will be automatically reloaded at start up and then continuously after a certain time has passed (see reload-time). auto-reload yes
reload-time <number> 30 The number of minutes between automatic reloads. reload-time 60
save-path <path> ~/ The default path where articles shall be saved to. If an invalid path is specified, the current directory is used. save-path "~/Saved Articles"
podcast-auto-enqueue [yes/no] no If yes, then all podcast URLs that are found in articles are added to the podcast download queue. See below for more information on podcast support in newsbeuter. podcast-auto-enqueue yes
cleanup-on-quit [yes/no] yes If yes, then the cache gets locked and superfluous feeds and items are removed, such as feeds that can't be found in the urls configuration file anymore. cleanup-on-quit no
user-agent <user agent string> "" If set to a non-zero-length string, this value will be used as HTTP User-Agent header for all HTTP requests. user-agent "Lynx/2.8.5rel.1 libwww-FM/2.14"
refresh-on-startup [yes/no] no If yes, then all feeds will be reloaded when newsbeuter starts up. This is equivalent to the -r commandline option. refresh-on-startup yes
suppress-first-reload [yes/no] no If yes, then the first automatic reload will be suppressed if auto-reload is set to yes. suppress-first-reload yes
cache-file <path> "~/.newsbeuter/cache.db" This configuration option sets the cache file. This is especially useful if the filesystem of your home directory doesn't support proper locking (e.g. NFS). cache-file "/tmp/testcache.db"
confirm-exit [yes/no] no If set to yes, then newsbeuter will ask for confirmation whether the user really wants to quit newsbeuter. confirm-exit yes
include <path> n/a With this command, you can include other files to be interpreted as configuration files. This is especially useful to separate your configuration into several files, e.g. key configuration, color configuration, … include "~/.newsbeuter/colors"

Table 2. Available Operations

Operation Default key Description
quit q Quit the program or return to the previous dialog (depending on the context).
reload r Reload the currently selected feed.
reload-all R Reload all feeds.
mark-feed-read A Mark all articles in the currently selected feed read.
mark-all-feeds-read C Mark articles in all feeds read.
open Enter Open the currently selected feed or article.
save s Save the currently selected article to a file.
next-unread n Jump to the next unread article.
open-in-browser o Opens the URL associated with the current article.
toggle-source-view ^u Toggles between the HTML view and the source view in the article view.
toggle-article-read N Toggle the read flag for the currently selected article.
toggle-show-read-feeds l Toggle whether read feeds should be shown in the feed list.
show-urls u Show all URLs in the article in a list (similar to urlview).
open-search / Opens the search dialog. When a search is done in the article list, then the search operation only applies to the articles of the current feed, otherwise to all articles.
select-tag t Select tag.
clear-tag ^t Clear current tag.
enqueue e Add the podcast download URL of the current article (if any is found) to the podcast download queue (see below for more information on podcast support).
redraw ^l Redraw the screen.
help ? Runs the help screen.

Keys, as used in the bind-key configuration command, use a special syntax. Lowercase keys, uppercase keys and special characters are written literally. The Enter key is written as "enter", while the ESC key is written as "esc". The function keys F1 to F12 are written as "F1" to "F12". The Space key is written as "space". Key combinations with the Ctrl key, such as Ctrl-R, are written as ^r. Please be aware that all Ctrl-related key combinations need to be written in lowercase ^R for Ctrl-Shift-R does not work.

Example Configuration

# a comment
max-items        100 # such comments are possible, too
browser          links
show-read-feeds  no
unbind-key       R
bind-key         ^r    reload-all

Configuring Colors

It is possible to configure custom color settings in newsbeuter. The basic configuration syntax is:

color <element> <foreground color> <background color> [<attribute> ...]

This means that if you configure colors for a certain element, you need to provide a foreground color and a background color as a minimum. The following colors are supported:

  • black
  • red
  • green
  • yellow
  • blue
  • magenta
  • cyan
  • white
  • default

The "default" color means that the terminal's default color will be used.

Optionally, you can also add one or more attributes. The following attributes are supported:

  • standout
  • underline
  • reverse
  • blink
  • dim
  • bold
  • protect
  • invis

Currently, the following elements are supported:

  • listnormal: a normal list item
  • listfocus: the currently selected list item
  • info: the info bars on top and bottom
  • background: the application background
  • article: the article text

The default color configuration of newsbeuter looks like this:

background   white   black
listnormal   white   black
listfocus    yellow  blue   bold
info         yellow  blue   bold
article      white   black

Advanced Features

Tagging

Newsbeuter comes with the possibility to categorize or "tag", as well call it, RSS feeds. Every RSS feed can be assigned 0 or more tags. Within newsbeuter, you can then select to only show RSS feeds that match a certain tag. That makes it easy to categorize your feeds in a flexible and powerful way.

Usually, the ~/.newsbeuter/urls file contains one RSS feed URL per line. To assign a tag to an RSS feed, simply attach it as a single word, separated by blanks such as space or tab. If the tag needs to contain spaces, you must use quotes (") around the tag (see example below). An example ~/.newsbeuter/urls file may look like this:

http://blog.fefe.de/rss.xml?html interesting conspiracy news "cool stuff"
http://rss.orf.at/news.xml news orf
http://www.heise.de/newsticker/heise.rdf news interesting

When you now start newsbeuter with this configuration, you can press "t" to select a tag. When you select the tag "news", you will see all three RSS feeds. Pressing "t" again and e.g. selecting the "conspiracy" tag, you will only see the http://blog.fefe.de/rss.xml?html RSS feed. Pressing "^t" clears the current tag, and again shows all RSS feeds, regardless of their assigned tags.

Scripts and Filters (Snownews Extensions)

From version 0.4 on, newsbeuter contains support for Snownews extensions. The RSS feed readers Snownews and Liferea share a common way of extending the readers with custom scripts. Two mechanisms, namely "execurl" and "filter" type scripts, are available and supported by newsbeuter.

An "execurl" script can be any program that gets executed and whose output is interpreted as RSS feed, while "filter" scripts are fed with the content of a configured URL and whose output is interpreted as RSS feed.

The configuration is simple and straight-forward. Just add to your ~/.newsbeuter/urls file configuration lines like the following ones:

exec:~/bin/execurl-script
filter:~/bin/filter-script:http://some.test/url

The first line shows how to add an execurl script to your configuration: start the line with "exec:" and then immediately append the path of the script that shall be executed. If this script requires additional parameters, simply use quotes:

"exec:~/bin/execurl-script param1 param2"

The second line shows how to add a filter script to your configuration: start the line with "filter:", then immediately append the path of the script, then append a colon (":"), and then append the URL of the file that shall be fed to the script. Again, if the script requires any parameters, simply quote:

"filter:~/bin/filter-script param1 param2:http://url/foobar"

In both cases, the tagging feature as described above is still available:

exec:~/bin/execurl-script tag1 tag2 "quoted tag"
filter:~/bin/filter-script:http://some.test/url tag3 tag4 tag5

A collection of such extension scripts can be found on this website: http://kiza.kcore.de/software/snownews/snowscripts/extensions

If you want to write your own extensions, refer to this website for further instructions: http://kiza.kcore.de/software/snownews/snowscripts/writing

Podcast Support

A podcast is a media file distributed over the internet using syndication feeds such as RSS, for later playback on portable players or computers. Newsbeuter contains support for downloading and saving podcasts. This support differs a bit from other podcast aggregators or "podcatchers" in how it is done.

Podcast content is transported in RSS feeds via special tags called "enclosures". Newsbeuter recognizes these enclosures and stores the relevant information for every podcast item it finds in an RSS feed. What the user then can do is to add the podcast download URL to a download queue. Alternatively, newsbeuter can be configured to automatically do that. This queue is stored in the file $HOME/.newsbeuter/queue.

The user can then use the download manager "podbeuter" to download these files to a directory on the local filesystem. Podbeuter comes with the newsbeuter package, and features a look and feel very close to the one of newsbeuter. It also shares the same configuration file.

Table 3. Podbeuter Configuration Commands

Configuration Command Argument(s) Default Description Example
download-path <path> ~/ Specifies the directory where podbeuter shall download the files to. download-path "~/Downloads"
max-downloads <number> 1 Specifies the maximum number of parallel downloads when automatic download is enabled. max-downloads 3
player <player command> "" Specifies the player that shall be used for playback of downloaded files. player "mp3blaster"

Table 4. Available Operations in Podbeuter

Operation Default key Description
quit q Quit the program.
pb-download d Download the currently selected URL.
pb-cancel c Cancel the currently selected download.
pb-delete D Delete the currently selected URL from the queue.
pb-purge P Remove all finished and deleted downloads from the queue and load URLs that were newly added to the queue.
pb-toggle-download-all a Toggle the "automatic download" feature where all queued URLs are downloaded one after the other. The "max-downloads" configuration option controls how many downloads are done in parallel.
pb-increase-max-dls + Increase the "max-downloads" option by 1.
pb-decrease-max-dls - Decrease the "max-downloads" option by 1. If the option is already 1, no further decrease is possible.

A usual "use case" is to configure newsbeuter to automatically enqueue newly found podcast download URLs. Then, the user reloads the podcast RSS feeds in newsbeuter, and after that, he/she uses podbeuter to view the current queue, and either selectively download certain files or automatically download them all together by pressing "a" within podbeuter.

Feedback

If you want to tell us something related to newsbeuter, don't hesitate to send an email: ak-newsbeuter@synflood.at

Alternatively, you can reach the newsbeuter developers on IRC: channel #newsbeuter on irc.freenode.net.

If you want to report newsbeuter bugs, please use this issue tracker: http://code.google.com/p/newsbeuter/issues/list

License

MIT/X Consortium License

©opyright 2006-2007 Andreas Krennmair <ak@synflood.at>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.