Dev/GettingStarted – Indico

Context Navigation

Version 75 (modified by gcerto, 2 years ago) (diff)
--

0. Preparation

You will need:

A computer running any modern Linux distribution or Windows (Mac OS X should work, although none of us have tested it). Most of the CERN Indico development team is currently using Ubuntu, so unless you have strong feelings otherwise, you might make your life easier by installing it.
The following software (Ubuntu package names listed in square brackets - use Synaptic Package Manager to find and install them)
- Python 2.xx (2.6 or greater advised) [python]
- Python development packages [python-dev]
- Apache httpd 2.2 or greater (make sure to use the pre-fork package!) [apache2-mpm-prefork]
- mod_wsgi 3.3 or greater [libapache2-mod-wsgi]
- git [git-core]
- gettext package [gettext]
- setuptools [python-setuptools]
- IPython [ipython] (needed if you want to run the Indico shell to manually query the database)
The following four packages will eventually be installed automatically by Indico, but for now you will need to install them yourself:
- PIL (version 1.1.7 already includes support for interlaced PNGs)
- ReportLab [python-reportlab]
- libxml2 [libxml2]
  - libxml2 Python bindings [python-libxml2]
- libxslt [libxslt1.1]
  - libxslt Python bindings [python-libxslt1]

For help installing the required software in Windows, see GitAndWindows.

1. Getting the source code

You should choose one of the following two options...

OPTION 1. If you are CERN staff/user, you should do the following:

Ask an administrator to create an account for you on indicodev.cern.ch
Make sure you have a CERN AFS account with space
Clone the git repo as follows:

For all examples in this document, let's pretend both your local username and indicodev account are jdoe

jdoe@localhost ~$ mkdir indico
jdoe@localhost ~$ cd indico
jdoe@localhost ~/indico$
jdoe@localhost ~/indico$ git clone ssh://jdoe@indicodev.cern.ch/afs/cern.ch/project/indico/repo/cds-indico.git
Initialized empty Git repository in /home/jdoe/indico/cds-indico/.git/

jdoe@indicodev.cern.ch's password: 
remote: Counting objects: 7854, done.
remote: Compressing objects: 100% (5497/5497), done.
remote: Total 7854 (delta 4368), reused 3252 (delta 2235)
Receiving objects: 100% (7854/7854), 26.73 MiB | 5627 KiB/s, done.
Resolving deltas: 100% (4368/4368), done.
jdoe@localhost ~/indico$

Note: If you want, you could set up AFS on your own PC and access it directly instead of ssh to indicodev.

OPTION 2. If you are not CERN staff/user:

You won't have access to the CERN AFS repository. You should clone the git repo using http as follows:
```
jdoe@localhost ~/indico$ git clone http://indico-software.org/repo/indico.git
```
If you make contributions that you would like to include in CERN's installation of Indico, read these instructions: Dev/Collaboration

Now, you should have a clone of the Indico source tree:

jdoe@localhost ~/indico$ cd cds-indico
jdoe@localhost ~/indico/cds-indico$ ls
AUTHORS  bin  ChangeLog  COPYING  doc  etc  ez_setup.py  indico  INSTALL  MANIFEST.in  NEWS  README  setup.py  tests  THANKS

2. Preparing the Development Environment

Indico requires some dependencies (libraries, etc...) that the setup.py installation script installs automatically, with the exception of ReportLab?, libxml2 and libxslt (which you have already installed manually). There's no need to worry about them, but if you want to know more about these dependencies, see Admin/Installation0.98.

Before running setup.py, you may want to switch to the most recent branch, for example v0.98-series. If you did not do this yet, you should still be in the "master" branch. You can verify this with:

jdoe@localhost ~/indico/cds-indico$ git status
# On branch master
nothing to commit (working directory clean)

To change to the most recent branch, first fetch it:

jdoe@localhost ~/indico/cds-indico$ git fetch origin v0.98-series:v0.98-series
From http://cdsware.cern.ch/repo/cds-indico
 * [new branch]      v0.98-series -> v0.98-series

and then switch (checkout) to it:

jdoe@localhost ~/indico/cds-indico$ git checkout v0.98-series

Finally, check that you are correctly in the target branch:

jdoe@localhost ~/indico/cds-indico$ git status
# On branch v0.98-series
nothing to commit (working directory clean)

Now you are ready to run the setup.py develop_config command. It will download several needed Python modules from the Web and create a development environment (as opposed to a production installation):

jdoe@localhost ~/indico/cds-indico$ python setup.py develop_config
running develop_config
Upgrading existing etc/indico.conf..

Indico needs to store some information in the filesystem (database, cache, temporary files, logs...)
Please specify the directory where you'd like it to be placed.
(Note that putting it outside of your sourcecode tree is recommended)
[/home/jdoe/indico/cds-indico]: /home/jdoe/indico
Creating directories... Done!
uid/gid not provided. Trying to guess them...  found apache(48) apache(48)

We need to 'sudo' in order to set the permissions of some directories...
Password:

Additionally, you should run:

jdoe@localhost ~/indico/cds-indico$ sudo python setup.py develop

So that setutools creates an egg-link in your global site-packages directory, creating a "fake egg" that will reside in your system-wide path.

You should edit the file .git/info/exclude, adding a list of patterns to match files you want git to ignore (you can add also the patterns to a .gitignore file at the root of the cds-indico folder). For example, you don't want emacs backup files being submitted to the git repo. Here is an example file you can use:

jdoe@localhost:~/indico/cds-indico$ cat .git/info/exclude
# git-ls-files --others --exclude-from=.git/info/exclude
# Lines that start with '#' are comments.
# For a project mostly in C, the following would be a good set of
# exclude patterns (uncomment them if you want to use them):
# *.[oa]
# *~


*.pyc
*.mo
*~
etc/indico.conf
etc/zdctl.conf
etc/zodb.conf

To check if there are more files that you want git to ignore, type git status:

jdoe@localhost:~/indico/cds-indico$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   indico/MaKaC/__init__.py
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	setuptools-0.6c11-py2.6.egg
no changes added to commit (use "git add" and/or "git commit -a")

In the above example, the file setuptools-0.6c11-py2.6.egg needs to be added to .git/info/exclude. The fact that this is being downloaded into cds-indico at all is a minor problem that may not get fixed any time soon

If you cannot get rid of a file even if you put its pattern in .git/info/exclude or .gitignore, you may want to use the following command:

git update-index --assume-unchanged indico/MaKaC/__init__.py

(use the path of the file that gives you a problem). However if you wish to commit a change later you will need to undo this with git update-index --no-assume-unchanged.

Set up mod_wsgi

From the version v0.98, Indico is running with mod_wsgi and not with mod_python anymore. If you already have a previous installation of Indico with mod_python, you should first remove it before starting the mod_wsgi installation and configuration. If this is your case, you should first take a look at the following page removing mod_python.

Once mod_python totally removed from your configuration you can proceed by installing mod_wsgi. The following steps will concern only the configuration in Linux, you can refer to this page to have the details about the configuration in Windows: Windows config.

Let's finally focus on the installation of mod_wsgi.

Create a new file in the 'sites-available' folder of apache. It's located by default under '/etc/apache2/sites-available/'.

jdoe@localhost ~$ sudo gedit /etc/apache2/sites-available/indico

Copy the next lines into that file, making sure to replace 'jdoe' with your username:

AddDefaultCharset UTF-8
<VirtualHost *:80>
        # mod_wsgi indico

	ErrorLog /var/log/apache2/error.log
        LogLevel warn
	
	Alias /indico/images "/home/jdoe/indico/cds-indico/indico/htdocs/images"
	Alias /indico/css "/home/jdoe/indico/cds-indico/indico/htdocs/css"
	Alias /indico/js "/home/jdoe/indico/cds-indico/indico/htdocs/js"
	Alias /indico/ihelp "/home/jdoe/indico/cds-indico/indico/htdocs/ihelp"

	WSGIDaemonProcess WSGIDAEMON processes=1 threads=1 inactivity-timeout=3600 maximum-requests=10000 \
            python-eggs=/home/jdoe/indico/tmp/egg-cache

	WSGIScriptAlias /indico "/home/jdoe/indico/cds-indico/indico/htdocs/index.wsgi"
	
	<Directory "/home/jdoe/indico/cds-indico/indico">
	    WSGIProcessGroup WSGIDAEMON
	    WSGIApplicationGroup %{GLOBAL}
	    AllowOverride None
	    Options None
	    Order deny,allow
	    Allow from all
	</Directory>
</VirtualHost>

Here's the explanation of the above lines:

Alias: Redirects some static locations to the containing folders, reducing load times.
WSGIDaemonProcess: Create 5 daemon processes of 1 thread each with name WSGIDAEMON. Set the python-path and python-eggs paths. (The other two parameters are for robustness).
WSGIScriptAlias: Redirect all petitions starting with /indico to the specified file.
WSGIProcessGroup: Configure the execution with the settings of WSGIDAEMON.
WSGIApplicationGroup: Set the execution to run under the same Python interpreter (the first created).

Finally, enable wsgi, the new webpage and restart apache.

jdoe@localhost ~$ sudo a2enmod wsgi
jdoe@localhost ~$ sudo a2ensite indico
jdoe@localhost ~$ sudo apache2ctl restart

Accessing http://localhost/indico/index.py should give you the main Indico page.

To have a better understanding of the WSGI interface you can refer to the following document Dev/Technical/WSGI. If you need more informations about the installation of v0.98, the configuration of a secured protocol and so on, please refer to Admin/Installation0.98

Edit the etc/indico.conf file.

Set the base URL

#------------------------------------------------------------------------------
# URLs
#------------------------------------------------------------------------------
# BaseURL is the url where Indico will be accessed:

BaseURL              = "http://myservername.cern.ch/indico"

# BaseSecureURL is the HTTPS location of Indico.

BaseSecureURL        = "https://myservername.cern.ch/indico"

Configure the SMTP server that Indico will use to send emails. Without this, you won't be able to create Indico users because Indico sends confirmation emails. You have 2 options:

Set SmtpServer to cernmx.cern.ch (will only work if your PC is inside CERN) and leave SmtpLogin and SmtpPassword blank (recommended), or ...

Set up a SMTP server on your PC and set SmtpServer to your PC. On Linux, you can easily set a SMTP server on your PC by installing the Postfix [postfix] package and selecting "Internet site" during its configuration. You can run "sudo dpkg-reconfigure postfix" or equivalent command to reconfigure it later if necessary.

#------------------------------------------------------------------------------
# SMTP
#------------------------------------------------------------------------------
SmtpServer           = "cernmx.cern.ch"
SmtpLogin            = ""
SmtpPassword         = ""

Set the web user and web user group to its correct value (e.g. apache, www-data).

#------------------------------------------------------------------------------
# APACHE
#------------------------------------------------------------------------------

# ApacheUser is the user that Apache runs as (Look for "User" inside httpd.conf)

ApacheUser = "www-data"

# ApacheGroup is the group that Apache runs as (Look for "Group" inside httpd.conf)

ApacheGroup = "www-data"

In order to use events' short URLs and RSS, folders /indico/events/ and /indico/categ/ have to be redirected by Apache. Add following lines inside your httpd.conf:

    RewriteEngine On
    RewriteRule ^/indico/event/(.*)$ /indico/events.py?tag=$1 [R=permanent,L]
    RewriteRule ^/indico/categ/(.*)$ /indico/categoryDisplay.py?categId=$1 [R=permanent,L]

or, if you don't want to use mod_rewrite:

   RedirectMatch permanent ^/indico/event/(.*)$ /indico/events.py?tag=$1
   RedirectMatch permanent ^/indico/categ/(.*)$ /indico/categoryDisplay.py?categId=$1

3. Launching Indico

To start the ZODB database, type:

jdoe@localhost:~/indico/cds-indico$ sudo zdaemon -C etc/zdctl.conf start

On Windows, some problems may occur while executing zdaemon. If so, try to type this instead:

runzeo -C etc/zodb.conf

If you are using Python 2.5, there may be problems creating the database. If so, contact indico-team@…! Also, you might be able to use the following script to create an empty database: Script to create an empty ZODB

Start or restart Apache httpd (any time you make changes to etc/indico.conf, remember to restart Apache again)

Open your browser and go to http://(your_host_name)/indico/index.py

Create an Indico user account for yourself, check your email and confirm

Log in with your new account, click on Server Admin, and under Server Administrator List, add the account you just created.

Since we have done "python setup.py develop" (and not "python setup.py install"), JavaScript menus won't work by default, for reasons that are too complicated for your pea-sized brain. To make them work, go to Server Admin, General Settings and check the Debug box. Isn't this fun?!

The Indico team at CERN likes to use the Firefox browser and the Firebug add-on for debugging web pages.

Room Booking module

If you need the room booking module (not activated by default) read the following: How to setup the room booking module for developers.

4. Working with Git

Before starting any development you should perform some basic steps with Git:

Setup your local Git

You can modify your .git/config file in order to setup your git environment. Here, you can see an example of config file.

Create a new local branch

You must create a new branch for each new project, bug fixing, improvement,etc. That branch must be created from a base one, for instance, if your contribution will be integrated in the official branch v0.98-series, you must create the new one based on that one, like this:

git checkout -b new-feature-A origin/v0.98-series

Working in your new branch

Every time you finish a sub-task related to your new-feature-A, you must commit it to your local repo.

If there are new files, do:

git add path/to/my/new/file.py

and now commit:

git commit -a

Write always a comment for every commit (see rules here)

Update your branch

If there are new contributions in your base branch (in our example, origin/v0.98-series) you can get them like this:

git fetch origin

and now rebase your branch with the updates (make sure you are currently in your branch new-feature-A):

git rebase origin/v0.98-series

NEVER rebase a branch that has already been published and integrated.

Once your new-feature-A is finished

You will need to do fetch and rebase, as explained in the previous point. And then you are ready to push:

git push your-public-repo new-feature-A

your-public-repo is a copy of your repo which is somewhere else and that the integrator can access to fetch and merge your new feature. In order to do that you must have these lines in your .git/config file:

[remote "your-public-repo"]
        url = myserver.com:/path/to/my/repo/indico-my-username.git
        fetch = +refs/heads/*:refs/remotes/your-public-repo/*

Once, it is published you can warn the integrators in order to merge your code.

5. Troubleshooting

Q: The first time I go to http://(your_host_name)/indico/index.py, I get a strange error involving a SyntaxError and a template.

A: Maybe someone messed up the template. Or maybe you did like me and downloaded the source code with !TortoiseCVS in Windows, and then copied it from Windows to Linux. Try doing a checkout again from Linux.

Q: When Indico tries to send an email, it doesn't work.

A: Try changing the SMTP server in MaKaCConfig.py to cernmx.cern.ch.

Q: What does MaKaC mean. I cannot sleep without knowing the answer'''

A: Once upon a time, an Indico elder told me that it was not a monkey's name. It actually meant 'Make a Conference'.

Q: I got an error similar to: cannot write to such and such file.

A: Try changing the permissions of the files so that Indico can write on them. If Indico is trying to create a file in a directory, Indico should have read, write and execution rights on that directory. Also create the directory if it doesn't exist already!

Q: I get a DisconnectedError when connecting to Indico for the first time.

A: This can happen in the following situation:

You are using python2.5 and (possibly) ZODB 3.4, 3.6, 3.7.
You don't have a data.fs file in the data directory, i.e., you are starting without a database file.

In this case, you have to create an empty database file with the following file (courtesy of Pedro):

import ZODB
from ZODB import FileStorage, DB
import transaction

storage = FileStorage.FileStorage("data.fs")
db = DB(storage)
connection = db.open()
dbroot = connection.root()

transaction.commit()

connection.close()
db.close()
storage.close()

Download in other formats:

Plain Text