Shells

Mu2e uses the bash linux shell exclusively for the interactive command line and shell scripting. In general, this is the most powerful shell and the professional's choice.

Setup scripts

When you log on to a unix-based machine your startup shell will automatically run commands located in certain "hidden files", some of which are located in your home directory and are editable by you. These are also known as login scripts. A general description of these files can be found on the relevant wikipedia page Because these files are hidden you cannot see them with a standard directory listing. To see them use ls -a ~/

If you are a bash expert and want to roll your own, read this section and make your own decisions. See also #Expert Only, below. If you are not a bash expert, follow the steps described here.

When you first get a Mu2e computing account, we recommend that you copy two example files, .bash_login and .bashrc, to your home directory. First, check to see if you have already have existing versions of these files:

> cd ~
> ls .bash_profile .bashrc
.bash_profile .bashrc

In the above, type everything to the right of "> " at the command prompt. The lines without a leading "> " are example ouput. If you have these files, make backup copies because the next steps will overwrite them:

> cd ~
> mv .bash_profile .bash_profile.sav  # only if needed
> mv .bashrc .bashrc.sav              # only if needed

Next, copy in the recommended files:

> curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bash_profile
> curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bashrc

In the above, the option argument to curl is a capital letter O, not the numeral zero. You can also click on the links to look at the files before your download them.

If you accidentally overwrite your existing files without making a copy, you can recover them from a snapshot of your home disk.

~/.bash_profile

• This file is executed by the system at the start of each login shell.
• If ~/.bashrc file exists, it sources ~/.bashrc. This ensures that a login shell has all of the functionality of a non-login shell. For all Mu2e computing work this is what you want.
• If you wish to supply your own customizations to ~/.bash_profile we suggest that you create a file named ~/.my_bash_profile and add your content to that file. The example .bash_profile looks for ~/.my_bash_profile and, if it exists, sources it. We recommend this so that you keep your customizations separate from the recommended base .bash_profile. This will simplify future evolution of the recommended base .bash_profile.
• If you had a previously existing .bash_profile, read the section #Obsolete setup patterns and then merge the still-useful content of your old .bash_profile into .my_bash_profile. Be careful not to needlessly duplicate content already present in your new .bash_profile.

~/.bashrc

• It defines the environment variable BASH_ENV to ensure that subshells have the same configuration as non-login shells. This is what you want for the Mu2e environment.
• It defines the alias mu2einit. When you run the command mu2einit, it creates the base Mu2e computing environment in your current shell.
• It sources /etc/bashrc which is needed to make available some Fermilab provided software.
• It enables two strongly recommended safety features, noclobber and ignoreeof. This also serves as an example of how to configure your environment differently for interactive and non-interactive shells.
• As for .bash_profile, we recommend that you put personal configuration in .my_bashrc.
• If you had a previously existing .bashrc, read the section #Obsolete setup patterns and then merge the still-useful content of your old .bashrc into .my_bashrc. Be careful not to needlessly duplicate content in your new .bashrc.
  • .my_bashrc is the file in which to put things like setting your prompt, defining aliases etc

We strongly recommend that you not run mu2einit in your login scripts. This is particularly important if you work on more than one Fermilab experiment. Experience has shown that you should keep your login scripts generic and define aliases, or functions, like mu2einit that create one experiment-specific environment. If you mix experiment-specific or project-specific environments, you will often get an inconsistent software configuration.

Other notes:

1. The Fermilab maintained systems execute /etc/profile.d/*.sh for login shells and /etc/bashrc for non-login shells. This may be different on other machines.
2. We recommend that you not use ~/.login, ~/.profile, ~/.shrc or ~/.cshrc . There is no reason for them to be present in a Mu2e environment. If they are not required by another project that you work on, we recommend you delete them.

Verify that Your Login Scripts Work Correctly

To verify that the login scripts work correctly, logout and log in again. Then do:

> mu2einit
> type muse
 muse is a function
 muse () 
 { 
     source ${MUSE_DIR}/bin/muse
 }
 

If you see the example output, the your login scripts are working correctly.

Pro Tip: you only need to type mu2ei followed by a tab and the command will auto-complete.

Obsolete setup patterns

In existing .bash_profile and .bashrc files, you may find code like the following:

pa=/grid/fermiapp/products/common/etc 
if [  -r "$pa/setups.sh" ]
then
  . "$pa/setups.sh"
  if ups exist login
  then
      setup login
  fi
fi

if [  -f "/afs/fnal.gov/ups/etc/setups.sh" ]
then 
    . "/afs/fnal.gov/ups/etc/setups.sh"
    if ups exist login
    then
        setup login
    fi
fi

upsfile="/cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups.sh"
if [  -r "${upsfile}" ]; then
    . "${upsfile}"
fi
unset upsfile

These are obsolete and should be removed.

Expert Only

If you choose that your login scripts will not define mu2einit, then everywhere that the Mu2e instructions tell you to

> mu2einit

you should instead:

> source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh

Retirement of UPS

During the spring of 2024 Mu2e will retire our use of UPS and will replace it with Spack. The old recommended login scripts used UPS to define the command setup mu2e that you used to create the Mu2e environment in your shell. You must update your login scripts to the new recommended scripts in order prepare for this transition. The old scripts will continue to work correctly until UPS is actually retired but we recommend that you update as soon as is convenient.

To update your login scripts, follow the instructions above at #Setup scripts. Be sure not to skip the steps that say to make copies of your existing files and to hand merge content that's still useful from your old files into your new files.

Once you have done this, you will need to change setup mu2e to mu2einit everywhere that you normally type it. Be sure to check your scripts as well.

The command mu2einit knows that we still using UPS on SL7 machines and that we using spack on AL9 machines.

Follow announcements on the Mu2e Slack channel "computing-and-software" to keep up to date with developments.

Other Startup Scripts

Many applications look in your home directory for a configuration script that is run at application-start-time and that lives in a hidden file in your home directory. Examples include editors and debuggers. This section discusses example configuration files for some of these products.

We invite everyone to propose improvements to the existing files and to propose example configurations for other applications. If you wish to contribute, speak with the Mu2e Computing management.

Editors

The two most widely used editors by current Mu2e members are emacs and vim. Both are installed on the Mu2e interactive machines so you do not need to install your own version. The hidden files that provide configuration for these editors are called .emacs and .vimrc, respectively.

Example files that conform to the Mu2e standards for formatting source code are available for both editors. We recommend that you check to see if you have an existing configuration file and save a copy before overwritting it with the Mu2e example file. For emacs users:

mv .emacs .emacs.sav # if needed
curl -O https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.emacs

For vim users:

 mv .vimrc .vimrc.sav # if needed
 curl -O https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.vimrc

Then merge any still-useful configuration from the saved files into the current ones. You can more about these files at: Editors.

gdb

gdb is the debugger provided along with the g++ compiler, the compiler used by Mu2e. There are two files used for the run-time configuration of gdb.

 mv .gdbinit .gdbinit.sav # if needed
 mv .gdb_stl .gdb_stl.sav # if needed
 https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.gdbinit
 https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.gdb_stl

You can read move about gdb at: CodeDebugging#gdb.

Beginner Cheat Sheet

This section summarizes, for beginners, some commonly used bash commands:

Files and directories

cp <x> <y> 	% copy <x> to <y>
mv <x> <y>	% move <x> to <y>
mkdir <x> 	% makes new directory <x>
pwd 		%shows current directory
cd <x>		% goes into folder <x>
rm <x> 		% remove file <x>
ln -s <x> <y>	% makes a soft link between real file <x> and local pointer <y>
ls 		% lists documents in current directory
cat <file> 	% list the whole file
more <file> 	% types <file> in chunks, <space> goes to next chunk
less <file>	% similar to more
head <file> 	% types the first N lines
tail <file>	% types the last N lines
find *		% list all of the files

Process management

CTL-Z	% pause the current process and return to console
bg 	% allow the process you just paused to run in background
jobs 	% shows what you're running
ps 	% shows processes 

<c> > file	% output of command c goes to file <file>
<c> >& file	% errors from command c goes to file <file>
<c> >> file	% appends output of <c> onto end of <file>
<c> >>& file	% appends error of <c> onto end of <file>

Help

man <c>	% help on command <c>
<c> -h	% sometimes has help this way as well

Editing and strings

sed s/<a>/<b>/ <file> > 
grep <a> <b>	% print out all lines in  <b> which contain string <a>
sort <a>	% sort the file a
diff <a> <b>	% print out the difference between <a> and <b>
xemacs <a>	% edit the file <a>

Environmentals

export <X>=<y> 		% will make $X refer to <y>
export PATH=<y>:${PATH} 	%will append <y> to $PATH

important environmentals are:

$HOST     % this computers's name
$USER    % your username
$HOME 		% your home area
$PWD 	            	% the current directory
$PATH 	            	% where unix looks for code to execute
$PYTHONPATH 	% where unix looks for python modules to import
$LD_LIBRARY_PATH	% where unix looks for shared libraries

You normally want to append to the PATH. Just setting them to <x> wipes out all the other stuff they were already set to.

Standards

These are the recommended practices for naming shell scripts that are intended to be used in the Mu2e Environment.

• scripts that intended to be executed by the average user at the command line, should be added to the user's path and can be named with no extension, so it looks built-in.
• scripts that are used in complex packages, or for special purposes, such as in monitoring, should have extension ".sh"
• scripts intended to be sourced should have the extension ".sh".

How Bash Startup Scripts Run

This section describes which of the shell startup scripts are sourced in which circumstances. It is correct for bash on SL7.

1. ssh hostname
   • This creates an interactive login shell that sources the following startup scripts
     • /etc/profile
       • This sources all of the files matching /etc/profile.d/*.sh
     • ~/.bash_profile, if present
     • If ~/.bash_profile is not present then source ~/.profile, if present
2. ssh hostname command
   • This creates a non-login shell that sources the following scripts
     • ~/.bashrc
       • Whether the shell is interactive or non-interactive depends on the command. If the command is something like "xterm" that expects input from a terminal, it will be interactive; if it is something like "ls /path", which does not expect input from a terminal, it will be non-interactive.
3. When you create a bash subshell (which is done implicitly when you run most commands that are not shell built-ins), the subshell will source
   • The file pointed to by $BASH_ENV, provided $BASH_ENV is a non-empty string.
4. When you create a bash subshell run in POSIX compatibility mode ( ie executed as /bin/sh, not as /bin/bash ) the subshell will source
   • The file pointed to by $ENV, provided $ENV is a non-empty string

The software deployed by Fermilab assumes that you have followed the recommendations below. If you do not, then some software installed by Fermilab may not work correctly.

1. Your ~/.bashrc should source /etc/bashrc
2. Your ~/.bash_profile should source your ~/.bashrc

Notes:

1. On mu2egpvm*, /bin/sh is a symlink to /bin/bash
2. On mu2egpvm*, /bin/slogin is a symlink to /bin/ssh
3. a method to determine what files a login shell is opening:

echo exit | strace bash -li |& grep '^open' > login.txt

References

The Linux Documentation Project publishes a number of guides that are available online. We suggest:

• Bash Guide for Beginners, a comprehensive introduction.
• BASH Programming - Introduction HOW-TO, a shorter but less complete introduction.
• Advanced Bash Scripting Guide, starts at the beginning put goes more quickly to advanced ideas.