Samuel's Project Blog

Grepping for stuff in MySQL dumps is not that nice, with miles-wide lines. You could send the grep output to a command such as "cut -c 1-200", but that would still not be guaranteed to give you the actual matched content.

Enter the "fold" command, which formats output into lines with a max count of chars:

grep "stuff" sqldump.sql | fold -w 200 | grep -C 1 "stuff"

... will give you a much better view of the context of the match!

(The first grep gets the (mile-wide) line that has the match, then fold will split the mile-wide line into 200 char long lines, and "grep -C 1" will show only the one 200 char wide line where the match is + 1 line of context before and after).

GNU Screen is a nice little program, allowing you to have "terminals" that you can detach in the background, so that you can have long batch jobs started, outputting stuff to the stdout, for example, but still don't be afraid to close down your terminal by accident etc.

Unfortunately screen has, IMO, quite an awkward syntax, but I managed to learn 3 flag combinations, and two keyboard combinations (from inside screen) that seems to be what I need for basic usage of screen:

Flags

Start a new named screen session:

screen -dmS ASessionName

screen -ls

screen -r ASessionName

Shortcuts

Detach the current session in background:

Ctrl + a, Ctrl + d

Ctrl + d

My work at UPPMAX, on the Bioclipse based HPC Client i is progressing, slowly but steadily. I just screencasted an experimental version of the job configuration wizard, which loads command line tool definitions from the Galaxy workbench, and use them to generate a GUI for configuring the parameters to the command line tool in question, as well as the parameters for the Slurm Resource manager (used at UPPMAX). Have a look if you want :) :

The Wizard obviously has quite some rough edges still. My current TODO is as follows:

• Set sensible default values i widgets (i.e. when there is just 1 alt)
• Use checkboxes and radiobuttons for select fields with few options
• Use progress bar between wizard pages that takes time to load
• Decide how to take care of the cheetah #if#lse#endif syntax, available in some galaxy tool config files.
• Add validation
• Use a time widget for the job time
• Add a custom view with just a "connect" button, and showing only remote files for the configured host.
• More modular loading of modules (hierarchical etc.)
• More advanced parsing of options (i.e. allowing to omit params, rather than just saying "no" on them).

Etc ... More suggestions? :)

Install xmlstarlet in Ubuntu:

sudo apt-get install xmlstarlet

Use the formatting command:

[code that produces some xml] | xmlstarlet fo

... or, if you are generating the XML with an XSLT stylesheet, don't forget the following line, after the xsl:stylesheet tag:

<xsl:output method="xml" indent="yes" encoding="UTF-8" />