apt-repo-tools

Readme


INTRO
-----
Repositories are often seen as a difficulty in Debian and its
derivatives. Whilst adding and removing packages has become easier and
easier, the same is not true for package repositories.

However, repositories are just text files in /etc/apt/sources.list.d,
and possibly a security key, both of which can be packaged.

Thus these tools aim to remove the complexities from managing
repositories by simply packaging up every repository. These can then be
installed, removed, updated, etc. just like any other package, with
absolutely no changes to the way the APT packaging system works. No need to
define arbitrary new protocols, distribution systems, system tools, etc. We just
use Debian's already excellent package facilities.

Since these tools are boring in themselves, also included are some scripts to
fetch data from apt-get.org and Launchpad's Personal Package Archives
(PPAs). This is stored in cache.txt and ppa.txt. Running make_sources.py
on these, followed by make_packages.py will generate over 8000 packages
in the temp folder, one for each repository. It's a start ;)

CONTENTS
--------
apt-get-org.py - Screen scrapes www.apt-get.org, a vast collection of
	third-party Debian repositories. Saves the data to a given file.

launchpad_ppa_grabber.py - Screen scrapes launchpad.net, site which hosts a few
	Ubuntu repositories

make_sources.py - Creates files suitable to put in sources.list.d.
	Automagically groups repos (eg. for matching deb and deb-src lines)

make_packages.py - Creates packages based on the contents of temp/lists

make_repo.py - Non-functional at the moment. Should make a repository
	out of repository packages.

cache-check.py - Does basic checks on a given file of repository data.

cache.txt - cached repo data (output from apt-get-org.py)

temp/ - Stores temporary files

temp/lists/ - Stores apt's sources.list files

temp/package/ - Stores temporary files for making packages

APT-GET-ORG.PY
--------------
This script takes an output filename "-o=something" (which will be
overwritten!) and fills it with repository data (in the format described
below) taken from www.apt-get.org.

It goes to the main page (http://www.apt-get.org/main/) and "screen
scrapes" data out of the page. It also gets package contents for each
repository by loading the info page for each one. Since this can
generate a load on their server, there is a half-second delay between
handling each repository.

MAKE_SOURCES.PY
---------------
This script is given an input file (in the format described below) with
the option "-f=something". It then parses this to find unique
combinations for repository/suite. Identical ones are grouped (eg.
"deb something.com stable" and "deb-src something.com stable") then
put into files suitable for including in /etc/apt/sources.list.d

MAKE_PACKAGES.PY
----------------
This script looks through the "temp/lists" folder for sources.list.d files
and generates a package for each one in "temp". The packages are
currently set to architecture "all", maintainer "unknown@unknown.com"
and have no dependencies or significant description.

CACHE-CHECK.PY
--------------
This script checks a given file of repository data (given with the
-f=filename option) using the format described below. Some basic
heuristics are used to validate the contents. This is useful since
screen scraping is prone to errors, which should preferably be spotted
ASAP rather than generating bogus packages.

Only the first error encountered is reported, since the parsing system
may fail afterwards. This means that a check-fix-check cycle must be
used until no errors are reported (and even then some obscure ones might
still be present).

DATA FORMAT
-----------
The format currently used to store repostory data is XML-esque with the
following structure

```

	
		An example.
		deb something.com sid
		deb something.com stable
		deb-src something.com sid
		deb-src something.com stable
		
			
			
		
		
			
			
			
		
		
			
		
	
	
		etc.
	

```

Repository

Clone this repository using:
  git clone http://chriswarbo.net/git/apt-repo-tools.git 

Branches


Generated by git2html.