Samuel's Project Blog

I often need to develop python scripts on some remote server where I can't run graphical python IDEs like spyder.

I'm too lazy to set up an advanced vim config with full blown IDE-like features (except for some basic python support).

I have found that a much simpler solution is a GNU screen session with two vertically split screens, one for the main coding in vim, and a smalelr one below, for running ipython, to run and debug the script as it is written.

I found it useful enough to figure out how to start such a setup with one command. This is how to do it:

create a file ~/.screenrc.pydev and place the following in it:

split
screen
focus
screen
resize 20
exec ipython
focus
exec vim

then add this to the bottom of your ~/.bashrc or ~/.bash_aliases:

alias pydev='screen -mS PyDev -c ~/.screenrc.pydev'

... and source the file:

source ~/.bashrc

Now, you can start your command line python environment by:

cd some-folder-with-python-files
pydev

The result:

Oh, and to jump between the screen windows, do:

[Ctrl] + [A], [Tab]

For formatting a 3-digit integer (that is, 2 leading zeros for 1-9):

printf %03d $i

... where $i is the number in question.

Grepping for stuff in MySQL dumps is not that nice, with miles-wide lines. You could send the grep output to a command such as "cut -c 1-200", but that would still not be guaranteed to give you the actual matched content.

Enter the "fold" command, which formats output into lines with a max count of chars:

grep "stuff" sqldump.sql | fold -w 200 | grep -C 1 "stuff"

... will give you a much better view of the context of the match!

(The first grep gets the (mile-wide) line that has the match, then fold will split the mile-wide line into 200 char long lines, and "grep -C 1" will show only the one 200 char wide line where the match is + 1 line of context before and after).

GNU Screen is a nice little program, allowing you to have "terminals" that you can detach in the background, so that you can have long batch jobs started, outputting stuff to the stdout, for example, but still don't be afraid to close down your terminal by accident etc.

Unfortunately screen has, IMO, quite an awkward syntax, but I managed to learn 3 flag combinations, and two keyboard combinations (from inside screen) that seems to be what I need for basic usage of screen:

Flags

Start a new named screen session:

screen -dmS ASessionName

screen -ls

screen -r ASessionName

Shortcuts

Detach the current session in background:

Ctrl + a, Ctrl + d

Ctrl + d

My work at UPPMAX, on the Bioclipse based HPC Client i is progressing, slowly but steadily. I just screencasted an experimental version of the job configuration wizard, which loads command line tool definitions from the Galaxy workbench, and use them to generate a GUI for configuring the parameters to the command line tool in question, as well as the parameters for the Slurm Resource manager (used at UPPMAX). Have a look if you want :) :

The Wizard obviously has quite some rough edges still. My current TODO is as follows:

• Set sensible default values i widgets (i.e. when there is just 1 alt)
• Use checkboxes and radiobuttons for select fields with few options
• Use progress bar between wizard pages that takes time to load
• Decide how to take care of the cheetah #if#lse#endif syntax, available in some galaxy tool config files.
• Add validation
• Use a time widget for the job time
• Add a custom view with just a "connect" button, and showing only remote files for the configured host.
• More modular loading of modules (hierarchical etc.)
• More advanced parsing of options (i.e. allowing to omit params, rather than just saying "no" on them).

Etc ... More suggestions? :)

Install xmlstarlet in Ubuntu:

sudo apt-get install xmlstarlet

Use the formatting command:

[code that produces some xml] | xmlstarlet fo

... or, if you are generating the XML with an XSLT stylesheet, don't forget the following line, after the xsl:stylesheet tag:

<xsl:output method="xml" indent="yes" encoding="UTF-8" />