Raindrop/Install: Difference between revisions
(→Notes: added "ERROR: module 'twisted>=8.2' has version conflict") |
(→Notes: link to http://getsatisfaction.com/mozilla_messaging/topics/strange_twisted_imap_error) |
||
Line 57: | Line 57: | ||
Notes for mac users: | Notes for mac users: | ||
# <tt>ERROR: module 'twisted>=8.2' has version conflict: (Twisted 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), Requirement.parse('twisted>=8.2'))</tt> [http://groups.google.com/group/raindrop/browse_thread/thread/2b85b96e0439a41c happens when trying to install twisted to the system python. Install Python 2.6 and make sure you're using it.] | # <tt>ERROR: module 'twisted>=8.2' has version conflict: (Twisted 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), Requirement.parse('twisted>=8.2'))</tt> [http://groups.google.com/group/raindrop/browse_thread/thread/2b85b96e0439a41c happens when trying to install twisted to the system python. Install Python 2.6 and make sure you're using it.] | ||
# doing easy_install PyOpenSSL after May 1st, may install | # doing easy_install PyOpenSSL after May 1st, may install some badly. It may be a PyOppenSSL 0.9 issue. 0.8 is known to work. Try the latest code, but if that does not work, switch to 0.8. | ||
some badly. It may be a PyOppenSSL 0.9 issue. 0.8 is known to work. Try | |||
the latest code, but if that does not work, switch to 0.8. | Twisted 0.9 (which is what you get if using easy_install or check-raindrop.py --configure) seems to cause problems with raindrop: [http://getsatisfaction.com/mozilla_messaging/topics/strange_twisted_imap_error]. | ||
== Running Raindrop == | == Running Raindrop == |
Revision as of 22:50, 23 December 2009
Check out the Hacking page for more context on the code.
Late Breaking News/Known Problems
- Integration with twisted logging is poor.
- We are in the middle of moving from using the manually executed 'run-raindrop.py' script to the couchdb managed 'couch-raindrop.py' script which is controlled by the front end. If both processes are running you can expect one to see regular conflict errors.
Installing Raindrop
couchdb
Install couchdb, version 0.10 or later. 0.9 will not work. See:
- http://couchdb.apache.org/downloads.html for most platforms.
- http://janl.github.com/couchdbx/ for a MacOSX installer
- http://wiki.apache.org/couchdb/Windows_binary_installer for Windows.
When building couchdb from source, it can be run without installing via a 'dev' install:
cd couchdb; ./bootstrap && ./configure && make && make dev ./utils/run
raindrop
- Install Python 2.5 or 2.6 (note that Python 3 is not yet supported).
- Install Mercurial (http://mercurial.selenic.com/)
- Checkout raindrop. For the 0.1 code that is shown on https://labs.mozilla.com/raindrop, use the 0.1 tag:
% hg clone -r 0.1 http://hg.mozilla.org/labs/raindrop
The trunk code is very much in flux, but if you prefer to use it, leave off the tag:
% hg clone http://hg.mozilla.org/labs/raindrop
- Execute the 'check-raindrop.py' script in the raindrop/server/python directory.
This should check all dependencies and let you know what you are missing, then check your couchdb for certain configuration requirements.
If you are brave, try executing 'check-raindrop.py --configure' (prefixing that command with 'sudo ' on some platforms). This will attempt to automatically install all dependencies necessary and configure couchdb.
If you are less brave, run it without the --configure option, then manually resolve any dependency or configuration issues it reports, then repeat until no problems are found.
A summary of the dependencies are: setuptools, twisted, PyOpenSSL, Skype4Py, python-twitter, simplejson, feedparser and paisley.
Notes
Notes for mac users:
- ERROR: module 'twisted>=8.2' has version conflict: (Twisted 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), Requirement.parse('twisted>=8.2')) happens when trying to install twisted to the system python. Install Python 2.6 and make sure you're using it.
- doing easy_install PyOpenSSL after May 1st, may install some badly. It may be a PyOppenSSL 0.9 issue. 0.8 is known to work. Try the latest code, but if that does not work, switch to 0.8.
Twisted 0.9 (which is what you get if using easy_install or check-raindrop.py --configure) seems to cause problems with raindrop: [1].
Running Raindrop
After check-raindrop.py reports everything is in order, the next step is to set up the accounts to use with raindrop, then have raindrop fetch the messages
Configuring
These steps should become less necessary as time moves forward (it will be possible in the future to do this work though a web interface), but for now, you need to set up the accounts you want to use in raindrop via a .raindrop file:
configure raindrop by editing ~/.raindrop
if your 'local' couchdb isn't on localhost, add a section along the lines of:
[couch-local] host=hostname port=port
Add one new section per account; note the "account-" prefix is important when defining accounts:
- Add gmail accounts along the lines of
[account-gmail-username] proto=imap kind=gmail username=username@gmail.com password=topsecret ssl=True
- Add twitter accounts along the lines of
[account-twitter-username] proto=twitter kind=twitter username=username password=topsecret
- Add imap accounts along the lines of
[account-imap-username] proto=imap kind=imap host=imap.example.com port=993 username=username password=topsecret ssl=True
- Add skype accounts along the lines of
[account-skype-username] proto=skype kind=skype username=username password=topsecret
- Add RSS feed along the lines of
[account-rss-feed] proto=rss uri=http://example.org/rss.xml
Accounts name must be unique. A 'rss' account already exists, which fetch messages from getsatisfaction.com
Once you are done with ~/.raindrop, then go to the command line, to the raindrop/server/python directory and run the following command:
% ./run-raindrop.py sync-messages --max-age=5days
It will take a few minutes for run-raindrop to complete. The above command gets the last 5 days of messages for each account configured above. Once the above command finishes, then open your favorite Firefox or WebKit-based browser and go to:
http://127.0.0.1:5984/raindrop/inflow/index.html
You should see some of your messages displayed.
There are many issues with the UI and how messages are processed, but hopefully you now have a feel for how raindrop works.
Controlling
The run-raindrop.py script is capable of fetching messages, re-running extensions, etc. It is expected that over time, this functionality will be rolled into the couchdb controlled couch-raindrop.py script. As a result, some functionality is duplicated in these scripts.
In particular, there is no protection against 'couch-raindrop.py' and run-raindrop.py attempting to perform the same thing at the same time, which will cause many errors to be logged. Before using the advanced features of run-raindrop, you should ensure couch-raindrop.py is not running by executing a command similar to:
% curl http://127.0.0.1:5984/raindrop/_raindrop/exit
When run-raindrop.py is executed, it always checks to see if the accounts and other documents required by raindrop (eg, the front-end applications) are up to date, then performs the requested operation, then terminates. If no operation is requested a web-browser pointing at raindrop will be opened. This means you can edit any of the files in the 'schema', 'client' or 'couch_doc' directories, then see those changes reflected in the front-end when run-raindrop is next started. We don't detect the changes while we are running though, only at startup.
Get messages from all accounts:
% ./run-raindrop.py sync-messages --max-age=2days --folder=inbox
Both --max-age and --folder are optional - see --help for details. Reload the front-end applications to see the messages as they arrive.
Unprocessing:
To delete all the intermediate messages in the DB, execute:
% ./run-raindrop.py unprocess
The next time you execute a 'process-backlog' command, all your messages will be reprocessed.
Error handling:
If there is an error during a 'process' operation (ie, an exception in the converter), we will write an 'error document' and continue processing the work queue. We don't make a very loud noise at the end when this happens - you need to notice it in the log output as it is running. At any time you can execute:
% ./run-raindrop.py retry-errors
To re-process those messages - but this will generally result in exactly the same error unless the converter has been fixed to deal with the condition causing the error. Alternatively, execute 'unprocess' to force reprocessing of all messages, not only those which previously caused an error.
Testing
Test everything using the test suite
% ./test-raindrop.py