All the users should know about AGAP HPC data center


Table of contents 


	Computing cluster
	Access in command line

	Ask a account
	Read the documentation
	Training
	How to connect
	How to manage data
	How to manage analyses
	Access from outside of the Cirad
	Report a problem and contact


	Access with Galaxy web workflow manager

	Ask a account
	Training / documentation
	How to connect
	How to manage data
	How to manage analyses
	Report a problem


	Access with R

	Training
	How to connect to R and RStudio on the computing cluster
	How to manage data
	How to run a job
	Contact


	Follow your project on Redmine
	Find other documentation on Alfresco Share

Computing cluster

Access in command line

Ask a account

Please fill the form.

Read the documentation

The User Charter of the computing cluster.

Some Operating Methods are available on the web site.

Training

To follow a training on Linux Command line / Job scheduler / Programming language, first have a look on our teaching sessions and read the document on our bioinformatics services (coming soon). For instance, look at course materials of the 2014 session.

How to connect

Serveur X11 et client ssh pour Windows

Installation Xming et Putty

Sur SourceForge Sourceforge, télécharger

Xming (serveur X11 pour windows)
Xming-fonts

Télécharger un client SSH (Putty est parfait)
Si vous utilisez putty, n'oublié pas de cocher la case "Enable X11 forwarding" dans les options SSH/X11
Pour changer les couleurs changer Default Foreground (noir), Default Bold Foreground (noir), Default Background (Blanc) et Default Bold Background (Blanc) dans les options Window/Colours

Vous pouvez utiliser le lien modify pour avoir la palette de couleurs

Si vous changer les couleurs après coup, n'oublier pas de sélectionner le serveur et faire load avant de changer les couleurs puis revenir à la session et save
Pour une meilleure gestion des fenêtres de connexion aux serveurs on peut ajouter au dessus de PuTTY, Manager

Ranger PuTTY par exemple dans C:\logiciel
Installer le setup puttycm0.6.0.4822beta.exe
Initialiser lechemin C:\logiciel\putty.exe au moment du lancement de PuTTY Connection Manager

Attention pour que cela fonctionne il ne faut pas mettre les login dans les Host Name de la configuration Putty par exemple cambresis.cirad.fr et non sbocs@cambresis.cirad.fr

Erreur d'utilisateur mettre le chemin C:\Program Files\PuTTY Connection Manager\puttycm.exe au lieu du chemin C:\logiciel\putty.exe
Alors là vous plantez l'application puttycm.exe et vous êtes bon pour appeler le 8855 pour qu'il utiliser regedit pour nettoyer tous les fichiers de puttycm.exe car la désinstallation/réinstallation redémarrage n'est pas suffisante pour que l'initialisation se passe de nouveau correctement.

Si vous perdez la fenêtre PuTTY sessions dans la fenêtre PuTTY Connection Manager, il suffit de faire clique droit sur la barre bleue de PuTTY Connection Manager puis choisir restaurer et l'affichage et redémarrer PuTTY Connection Manager.
PuTTY CM permet de résoudre le problème de umask 007.

Ensuite modifier le raccourci Xming qu'il y a dans le menu démarrer et ajouter l'argument -xkblayout fr, sinon vous aurez votre clavier en qwerty :

Démarrer
Tous les programmes
Xming
Click droit sur le programme Xming
Propriétés
Dans l'onglet Raccourci, rajouter Dans la case Cible -xkblayout fr

Ensuite et bien il n'y a plus qu'à cliquer sur le raccourci Xming pour que le serveur se lance.

Once you have ssh

$ ssh -Y marmadais.cirad.fr
SIDIBEBOCS@marmadais.cirad.fr's password:
Last login: Fri Mar 27 15:01:03 2015 from pcbat3-11.cirad.fr

################
################
Welcome on CIRAD Southgreen HPC platform.

Reminder :
Directory work (~/work) is NOT backed-up.

################
################

Pourquoi je n'arrive pas à me connecter avec Putty ?

nadir@ipowerht:~ $ ssh nadir@10.0.0.4
The authenticity of host '10.0.0.4 (10.0.0.4)' can't be established.
RSA key fingerprint is c2:72:14:de:97:a3:68:e6:80:96:e5:f6:03:6d:ab:b0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.4' (RSA) to the list of known hosts.
nadir@10.0.0.4's password:

Il faut répondre oui, si vous avez répondu non, il se peut que vous ne puissiez plus vous connecter
dans ce cas il faut effacer ou renommer le fichier .Xauthority dans le répertoire home du serveur auquel vous n'arrivez pas à vous connecter

$ mv .Xauthority old.Xauthority
login as: sbocs
sbocs@cambresis.cirad.fr's password:
/usr/X11R6/bin/xauth:  creating new authority file /home/sbocs/.Xauthority

voir aussi éventuellement le fichier qui se trouve dans le répertoire .ssh

Pourquoi je n'arrive pas à faire un copier coller avec le bouton du milieu ?

Avec Putty, le copier-coller est fait avec un clic droit et non un clic milieu.

Installation de mobaxterm

Il fait à la fois serveur X11 et clien ssh tabulé. Le télécharger.

Serveur X11 et client ssh pour Linux

Installation Xming et Putty

Pour passer par KDE, lancer X-minglaunch,

Quitter xming si il est ouvert
Démarrer
Tous les programmes
X-ming
XLaunch
Sélectionner "one window"*
Suivant
Sélectionner "start no client"
Suivant
Dans la case "Additional parameters for Xming" rajouter "-xkblayout fr"
sauvegarder config.xlaunch
Terminer

Lancer Putty et se connecter sur Valois. Startkde ne se trouve que sur valois. Dans le terminal, lancer KDE, '$ startkde'.
Si vous voulez relancer cette configuration de xming

double cliquer sur config.xlaunch

How to manage data

Transfer data

scp command on Mac and Linux system

$ scp sidibebocs@cc2-login.cirad.fr:/home/sidibebocs/toto .

scp command between 2 servers

[sidibebocs@cc2-login ~]$ scp toto sidibebocs@alberes.cirad.fr:~/

Filezilla in all cases and choose the sftp option

Create and Edit files

You can use nedit but prefer Notepad++ for Windows or textwrangler for Mac

$ nedit toto&
[1] 30003
$ nedit: the current locale is utf8 (en_US.UTF-8)
nedit: changed locale to non-utf8 (en_US)

Rights on files

$ ls -l toto
-rw-rw---- 1 sidibebocs cirad 5 Mar 28 23:10 toto
[1]+  Done                    nedit toto
$ chmod a+r toto
$ ls -l toto
-rw-rw-r-- 1 sidibebocs cirad 5 Mar 28 23:10 toto

Follow your quotas

Read the best practices (in french)

You can use this command to see the disk usage

$ df -h
Filesystem                 Size  Used Avail Use% Mounted on
cc-nas:/ifs/hom            87T   65T   19T  79% /NAS/home
cc-nas:/ifs/Genomics        5.5T  5.2T  364G  94% /NAS/Genomics
cc-nas:/ifs/arcad_data     14T   13T  1.4T  91% /NAS/arcad_data
cc-nas:/ifs/davem_data      8.0T  6.8T  1.3T  85% /NAS/davem_data
cc-nas:/ifs/NGS             5.0T  2.7T  2.4T  54% /NAS/NGS
cc-nas:/ifs/GBS             3.5T  2.8T  809G  78% /NAS/GBS
cc-nas:/ifs/ESTtik        500G  317G  184G  64% /NAS/cambresis/ESTtik
cc-nas:/ifs/BAILLARGUET    10T  3.5T  6.6T  35% /NAS/BAILLARGUET
cc-nas:/ifs/OPGP_data      87T   65T   19T  79% /NAS/OPGP_data
cc-nas:/ifs/greenphyl       1.0T  520G  505G  51% /NAS/greenphyl
cc-nas-backup:/ifs/galaxy 130T  111T   16T  89% /NAS_BCK/galaxy
/dev/gpfs_sas               9.9T  9.7T  204G  98% /work
/dev/gpfs_sata             11T  8.6T  2.4T  79% /SATA

The quota -v command does not work so you can use

$ du -sh sidibebocs
17G    sidibebocs

Know your unix groups

$ groups
cirad greenphyl musatract coffea arcad esttik

Remove a lot of small files

Read the best practices (in french).

Work with compressed file

Read the tutorial.

How to manage analyses

Be sure that your programme is executable

$ ls -l count_target.pl
-rw-r-x--- 1 sidibebocs cirad 1649 Mar 23  2011 count_target.pl
$ chmod a+x count_target.pl
$ ls -l count_target.pl
-rwxr-x--x 1 sidibebocs cirad 1649 Mar 23  2011 count_target.pl

Run a job

Read the procedure (in french) to submit a job on SGE scheduler and also a basic help on SGE.

$ qsub -q bioinfo.q -b Y -V -N testa /home/tisne/work/Simwalk2/a.out

Sur cc2 par défaut 4Go par job
une option memfree

$ qsub -q normal.q -l mem_free=16

Ask a new software installation

First look at the list of avaible software on the computing cluster, then if it is not present ask for a new installation filling the form.

Share your sources codes

You can use Git versionning for instance have a look on the South Green Github, the T. Flutre Github or on AGAP Gitorious.

Know your authorized queues

Read the list of available queues on the computing cluster.

You can use

$ qstat -g c
CLUSTER QUEUE                   CQLOAD   USED    RES  AVAIL  TOTAL aoACDS  cdsuE  
--------------------------------------------------------------------------------
arcad.q                           0.09      0      0    176    184      0      8
bigmem.q                          0.00      9      0     15     24      0      8
bioinfo.q                         0.09      3      0    173    184      0      8
esttik.q                          0.09      0      0    176    184      0      8
gnpannot.q                        0.09      0      0    176    184      0      8
greenphyl.q                       0.09      0      0    176    184      0      8
hypermem.q                        -NA-      0      0      0     32      0     32
long.q                            0.09      9      0    167    184      0      8
normal.q                          0.09     16      0    160    184      0      8
on_the_fly.q                      0.09      0      0    176    184      0      8
urgent.q                          0.09      0      0    192    240      0     48
web.q                             0.09      5      0    187    208      0     16

Finally authorized queues for a user is notified in the mail with personnal identifiers to log on the computing cluster (coming soon).

Access from outside of the Cirad

First you should ask a vpn account filling the 8855 form; then you should use Forticlientsslvpn available here.

Report a problem and contact

To report a problem, fill the form.

For any other question about access of the computing cluster write to admin.bioinfo@cirad.fr.

Access with Galaxy web workflow manager

Ask a account

Please fill the form.

Training / documentation

To Follow a training on Galaxy, first have a look on our teaching sessions and read the document on our bioinformatics services (coming soon). For instance, look at course materials of the 2014 session.

You can also look general documentation at https://wiki.galaxyproject.org/Support

How to connect

Go to the following address: http://galaxy.southgreen.fr/galaxy/ and then login

How to manage data

You can upload your datafile, use shared libraries, follow your diskusage, delete files or histories, share your histories; follow the course but also have a look on the published pages.

How to manage analyses

You can execute a tool, run, share or publish a workflow see the exercices.

Ask a new software installation and conctact

Make your request to galaxy-dev-southgreen@cirad.fr

Report a problem

To report a problem, fill the form.

Access with R

Training

To follow a training on R, first you can read Introduction to R document published by Julien Barnier (http://alea.fr.eu.org/pages/intro-R).

You can look at http://www.biocampus.cnrs.fr/index.php/fr/ateliers-technologiques

Statistiques de base pour la génomique
Analyse de données RNA-seq sous R

You can also have a look on first have a look on our teaching sessions.

How to connect to R and RStudio on the computing cluster

You can just run R on your xterm console

$ R
R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)
R est un logiciel libre livré sans AUCUNE GARANTIE.
Vous pouvez le redistribuer sous certaines conditions.
Tapez 'license()' ou 'licence()' pour plus de détails.
R est un projet collaboratif avec de nombreux contributeurs.
Tapez 'contributors()' pour plus d'information et
'citation()' pour la façon de le citer dans les publications.
Tapez 'demo()' pour des démonstrations, 'help()' pour l'aide
en ligne ou 'help.start()' pour obtenir l'aide au format HTML.
Tapez 'q()' pour quitter R.
> 3+2
[1] 5

You can run Rstudio and login with your cluster identifiers.
http://cc2-web1.cirad.fr/rstudio/auth-sign-in
http://marmadais.cirad.fr/rstudio/ is obsolete

[sidibebocs@cc2-login homedir]$ module load compiler/gcc/4.9.2
[sidibebocs@cc2-login homedir]$ module load bioinfo/gdal/1.9.2
[sidibebocs@cc2-login homedir]$ module load bioinfo/geos/3.4.2
[sidibebocs@cc2-login homedir]$ module load bioinfo/R/3.1.3
[sidibebocs@cc2-login homedir]$ R
R version 3.1.3 (2015-03-09) -- "Smooth Sidewalk"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
  Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(Matrix)
> library(MASS)

It is also possible to modify your .bash_profile in you homedirectory: add the default modules to load

[root@cc2-login ~]# more /home/jacquin/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
# New environment setting added by IBM System Storage DS Storage Manager 10 on Tue Jan 26 14:47:03 CET 2010 1.
# The unmodified version of this file is saved in /root/.bash_profile1769444710.
# Do NOT modify these lines; they are used to uninstall.
PATH="${PATH}:/opt/IBM_DS/jre/bin"
export PATH
module load compiler/gcc/4.9.2
module load bioinfo/gdal/1.9.2
module load bioinfo/geos/3.4.2
module load bioinfo/R/3.1.3
# End comments by InstallAnywhere on Tue Jan 26 14:47:03 CET 2010 1.

Problem: "I can not run R Studio from bigmen". "Resuming R session ..." then "error navigating to", code 502.

Have a look in the temporary files of rstudio for instance remove or rename persistent-state

/homedir/sidibebocs/.rstudio
[sidibebocs@cc2-login .rstudio]$ more persistent-state
abend="0"
active-client-id="96ba65c9-5b8d-4090-833a-20510f256c50"
build-last-errors="[]"
build-last-errors-base-dir=""
build-last-outputs="[]"
compile_pdf_state="{\"errors\":[],\"output\":\"\",\"running\":false,\"tab_visible\":false,\"target_file\":\"\"}"
console_procs="[]"
files.monitored-path=""
find-in-files-state="{\"handle\":\"\",\"input\":\"\",\"path\":\"\",\"regex\":true,\"results\":{\"file\":[],\"line\":[],\"lineValue\":[],\"matchOff\":[],\"matchOn\":[]},\"running\":false}"
imageDirtyState="1"
saveActionState="-1"

How to manage data

First, you can load a package

library(qtl)

Then, you can set the working directory

setwd("~/Formation")

Finally, load your file

mycro<-read.cross(format="csv",file="f2300-GYLD.csv",sep=",")

How to run a job

For instance

plot(mycro)

Exemple of command line to run a R script (coming soon)

Contact

Ask a new package installation or report a problem to admin.bioinfo@cirad.fr

Follow your project on Redmine

http://redmine.southgreen.fr/projects

Find other documentation on Alfresco Share

https://collaboratif.cirad.fr/share/page/site/AGAPsgbpID/dashboard

Partners	Funding partners	Networks

Secondary menu

All the users should know about AGAP HPC data center

Computing cluster

Access in command line

Ask a account

Read the documentation

Training

How to connect

Serveur X11 et client ssh pour Windows

Installation Xming et Putty

Once you have ssh

Installation de mobaxterm

Serveur X11 et client ssh pour Linux

Installation Xming et Putty

How to manage data

Transfer data

Create and Edit files

Rights on files

Follow your quotas

Know your unix groups

Remove a lot of small files

Work with compressed file

How to manage analyses

Be sure that your programme is executable

Run a job

Ask a new software installation

Share your sources codes

Know your authorized queues

Access from outside of the Cirad

Report a problem and contact

Access with Galaxy web workflow manager

Ask a account

Training / documentation

How to connect

How to manage data

How to manage analyses

Ask a new software installation and conctact

Report a problem

Access with R

Training

How to connect to R and RStudio on the computing cluster

How to manage data

How to run a job

Contact

Follow your project on Redmine

Find other documentation on Alfresco Share

Main menu

Platform