NAME
App::CPANIDX - Queryable web-based CPAN Index
SYNOPSIS
# Generate the index database
$ cpanidx-gendb --config cpanidx.ini
# Run the FastCGI script
$ cpanidx-fcgi --config cpanidx.ini
DESCRIPTION
App::CPANIDX provides a number of scripts to build a queryable web-based
CPAN index.
CONFIGURATION
Configuration is dealt with by a Config::Tiny based configuration file.
There are a number of parameters which can be specified
"dsn"
The DBI dsn string of the database that the scripts will use. This
is a mandatory requirement.
"user"
The username for the supplied "dsn".
"pass"
The password for the supplied "dsn".
"url"
The "cpanidx-gendb" script will poll this url when it has finished
its update. It should be the root url of your CPANIDX site
url=http://my.cpanidx.site/cpanidx/
"mirror"
The url of a CPAN mirror site where "cpanidx-gendb" will obtain its
index files from. If not supplied it defaults to the Funet site
.
"socket"
This is the socket that FCGI should listen on for requests. It is a
mandatory requirement for the "cpanidx-fcgi" script.
"skipcore"
Applicable to the "cpanidx-gendb" script, will skip the generation
of the Module::CoreList based tables.
"skipmirrors"
Applicable to the "cpanidx-gendb" script, will skip the generation
of the mirrorlist based tables.
"skipperms"
Applicable to the "cpanidx-gendb" script, will skip the generation
of the CPAN permissions based tables.
SCRIPTS
Both the scripts will by default look for a "cpanidx.ini" file in the
current working directory unless you specify an alternative with the
"--config" command line option.
"cpanidx-gendb"
Generates the CPANIDX database. It will retrieve the CPAN index
files from a CPAN mirror and parse them to build the database.
The CPAN indexes are downloaded to "~/.cpanidx" by default. You may
override this location by setting the "PERL5_CPANIDX_DIR"
environment variable to a different location to use.
In tests a DBD::SQLite database took over 3 minutes to generate and
a DBD::mysql database took 30 seconds.
It is recommended that one uses cron or some such scheduler to run
this script every hour to ensure freshness of the CPAN index.
"cpanidx-fcgi"
Presents the CPAN index to web clients via FastCGI. Specify a socket
that the script should listen for requests on and configure your
webserver accordingly.
The following is an example for Lighttpd:
fastcgi.server = (
"/cpanidx/" =>
( "localhost" => (
"host" => "127.0.0.1",
"port" => 1027,
"check-local" => "disable",
)
),
)
The interface that clients can query is described below.
INTERFACE
The "cpanidx-fcgi" provides a number of ways that clients can access
information from the CPAN Index.
The information is provided in a number of different formats: YAML,
JSON, XML and HTML.
Information is requested by using a special URL
http://name.of.website////
We will assume that is "cpanidx" for the purposes of this
documentation.
"
The format may be one of "yaml", "json", "xml" or "html".
"
The command may be one of the following:
"mod"
Takes a search term which is a module name to search for.
Returns information relating to that module if it exists.
curl -i http://name.of.website/cpanidx/yaml/mod/LWP
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 04 Mar 2010 11:34:07 GMT
Server: lighttpd/1.4.25
---
-
cpan_id: GAAS
dist_file: G/GA/GAAS/libwww-perl-5.834.tar.gz
dist_name: libwww-perl
dist_vers: 5.834
mod_name: LWP
mod_vers: 5.834
"dist"
Takes a search term which is a distribution name to search for.
Returns information relating to that distribution if it exists.
curl -i http://name.of.website/cpanidx/yaml/dist/CPANPLUS-Dist-Build
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Mon, 06 Sep 2010 14:02:23 GMT
Server: lighttpd/1.4.25
---
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/CPANPLUS-Dist-Build-0.48.tar.gz
dist_name: CPANPLUS-Dist-Build
dist_vers: 0.48
"auth"
Takes a search term which is the CPAN ID of an author to search
for. Returns information relating to that author if they exist.
curl -i http://name.of.website/cpanidx/yaml/auth/BINGOS
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 04 Mar 2010 11:36:13 GMT
Server: lighttpd/1.4.25
---
-
cpan_id: BINGOS
email: chris@bingosnet.co.uk
fullname: 'Chris Williams'
"dists"
Takes a search term which is the CPAN ID of an author. Returns a
list of distributions that author has on CPAN.
curl -i http://name.of.website/cpanidx/yaml/dists/BINGOS
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 04 Mar 2010 11:39:14 GMT
Server: lighttpd/1.4.25
---
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/POE-Filter-LZO-1.70.tar.gz
dist_name: POE-Filter-LZO
dist_vers: 1.70
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/POE-Component-Server-SimpleSMTP-1.44.tar.gz
dist_name: POE-Component-Server-SimpleSMTP
dist_vers: 1.44
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/POE-Component-Server-RADIUS-1.02.tar.gz
dist_name: POE-Component-Server-RADIUS
dist_vers: 1.02
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/Archive-Extract-0.38.tar.gz
dist_name: Archive-Extract
dist_vers: 0.38
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/POE-Component-IRC-Plugin-URI-Find-1.08.tar.gz
dist_name: POE-Component-IRC-Plugin-URI-Find
dist_vers: 1.08
-
cpan_id: BINGOS
dist_file: B/BI/BINGOS/POE-Component-SmokeBox-Dists-1.00.tar.gz
dist_name: POE-Component-SmokeBox-Dists
dist_vers: 1.00
etc, etc.
"perms"
Takes a search term which is a module name to search for.
Returns CPAN permissions relating to that module if it exists.
The permission is one of "m" for "modulelist", "f" for
"first-come" and "c" for "co-maint".
curl -i http://name.of.website/cpanidx/yaml/perms/POE::Component::SmokeBox::Dists
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 04 Mar 2010 11:39:14 GMT
Server: lighttpd/1.4.25
---
-
mod_name: POE::Component::SmokeBox::Dists
cpan_id: BINGOS
perms: f
"timestamp"
Does not take a search term. Returns a timestamp of when the
CPAN Index Database was last updated and when the packages file
that was used was last updated. Both values are in epoch time.
curl -i http://name.of.website/cpanidx/yaml/timestamp
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Wed, 09 Jun 2010 10:16:15 GMT
Server: lighttpd/1.4.25
---
-
lastupdated: 1276075625
timestamp: 1276077865
"topten"
Does not take a search term. Returns a list of the authors with
the most distributions. This is not the most accurate, try
for a more
accurate leaderboard.
curl -i http://name.of.website/cpanidx/yaml/topten
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 04 Mar 2010 11:44:44 GMT
Server: lighttpd/1.4.25
---
-
cpan_id: ADAMK
dists: 237
-
cpan_id: RJBS
dists: 215
-
cpan_id: ZOFFIX
dists: 212
-
cpan_id: MIYAGAWA
dists: 190
-
cpan_id: SMUELLER
dists: 130
-
cpan_id: NUFFIN
dists: 122
-
cpan_id: TOKUHIROM
dists: 121
-
cpan_id: BINGOS
dists: 121
-
cpan_id: GUGOD
dists: 118
-
cpan_id: MARCEL
dists: 114
"mirrors"
Does not take a search term. Returns a list of CPAN mirror sites
as listed in the "MIRRORED.BY" file.
curl -i http://name.of.website/cpanidx/yaml/mirrors
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Mon, 19 Apr 2010 14:52:52 GMT
Server: lighttpd/1.4.25
---
-
dst_bandwidth: STM-1
dst_contact: tenet.ac.za^aa
dst_ftp: ftp://cpan.mirror.ac.za/
dst_http: http://cpan.mirror.ac.za/
dst_location: 'Cape Town, South Africa, Africa (-33.93 18.47)'
dst_notes: ''
dst_organisation: TENET
dst_rsync: mirror.ac.za::cpan
dst_src: rsync://www.cpan.org/CPAN/
dst_timezone: '+2'
frequency: '12 Hourly'
hostname: mirror.ac.za
-
dst_bandwidth: 50MB
dst_contact: is.co.za|ftpadmin
dst_ftp: ftp://ftp.is.co.za/pub/cpan/
dst_http: http://mirror.is.co.za/pub/cpan/
dst_location: 'Johannesburg, Gauteng, South Africa, Africa (-26.17 28.03)'
dst_notes: 'Limit to 4 simultaneous connections.'
dst_organisation: 'Internet Solutions'
dst_rsync: ftp.is.co.za::IS-Mirror/ftp.cpan.org/
dst_src: rsync.nic.funet.fi
dst_timezone: '+2'
frequency: daily
hostname: is.co.za
-
dst_bandwidth: T3
dst_contact: saix.net=ftp
dst_ftp: ftp://ftp.saix.net/pub/CPAN/
dst_http: ''
dst_location: 'Parow, Western Cape, South Africa, Africa (-33.9064 18.5631)'
dst_notes: ''
dst_organisation: 'South African Internet eXchange (SAIX)'
dst_rsync: ''
dst_src: ftp.funet.fi
dst_timezone: '+2'
frequency: daily
hostname: saix.net
-
etc. etc.
"corelist"
Takes a search term which is a module name to search for.
Returns information if that module is shipped with perl core.
curl -i http://name.of.website/cpanidx/yaml/corelist/Class::ISA
HTTP/1.1 200 OK
Content-type: application/x-yaml; charset=utf-8
Transfer-Encoding: chunked
Date: Thu, 06 May 2010 09:31:25 GMT
Server: lighttpd/1.4.25
---
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.007003
released: 2002-03-05
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008
released: 2002-07-19
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008001
released: 2003-09-25
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008002
released: 2003-11-05
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008003
released: 2004-01-14
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008004
released: 2004-04-21
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008005
released: 2004-07-19
-
deprecated: 0
mod_vers: 0.32
perl_ver: 5.008006
released: 2004-11-27
etc etc.
AUTHOR
Chris "BinGOs" Williams
LICENSE
Copyright © Chris Williams
This module may be used, modified, and distributed under the same terms
as Perl itself. Please see the license that came with your Perl
distribution for details.
SEE ALSO
Config::Tiny
DBI
FCGI