Shells: Difference between revisions
Line 4: | Line 4: | ||
==Setup scripts== | ==Setup scripts== | ||
These are also known as login scripts. | |||
If you are a bash expert, read this and make your own decisions. See also [[#Expert Only]], below. | |||
If you are not a bash expert, follow the steps described here. | |||
When you first get a Mu2e computing account, we recommend that you copy two example files, .bash_login and .bashrc, to your home directory. | |||
The Mu2e computing environment, and the Mu2e documentation, presume that you setup [[UPS]] in your login script and these login scripts do that. You will see the use of the UPS system in the tutorials and we will describe it as you use it. The short version is that UPS is a system that makes additional software available in your working shell; you can choose to create different environments in different shell windows. If you have a lot of time you can read about [[UPS]] in advance but it's not necessary. | |||
First, check to see if you have already have existing versions of these files: | |||
> cd ~ | |||
> ls .bash_profile .bashrc | |||
.bash_profile .bashrc | |||
If you have these files, make backup copies because the next steps will overwrite them: | |||
> cd ~ | |||
> mv .bash_profile .bash_profile.sav # only if needed | |||
> mv .bashrc .bashrc.sav # only if needed | |||
> curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bash_profile | |||
> curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bashrc | |||
In the above, the option argument to curl is a capital letter O, not the numeral zero. You can also click on the links to look at the files before your download them. | |||
Here are some comments about each of these files. | |||
* Example [https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.bash_profile ~/.bash_profile] | * Example [https://github.com/Mu2e/Bootstrap/blob/main/dotFiles/.bash_profile ~/.bash_profile] | ||
** This file is executed at the start of each login shell. | ** This file is executed at the start of each login shell. | ||
** It sources /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups.sh, which initializes the [[UPS]] system for use in your shell. | ** It sources /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups.sh, which initializes the [[UPS]] system for use in your shell. | ||
** The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bash_profile. The example .bash_profile checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bash_profile. | ** The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bash_profile. The example .bash_profile checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bash_profile. | ||
** If you had a previously existing .bash_profile, you should merge its content in .my_bash_profile. Be careful not to needlessly duplicate content in your new .bash_profile. | |||
** It sources ~/.bashrc, which is not executed by the system for login shells. | ** It sources ~/.bashrc, which is not executed by the system for login shells. | ||
** It defines BASH_ENV to point to ~/.bashrc so that is run for non-interactive shells | ** It defines BASH_ENV to point to ~/.bashrc so that is run for non-interactive shells | ||
Line 20: | Line 38: | ||
** It sources /etc/bashrc for system settings | ** It sources /etc/bashrc for system settings | ||
** The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bashrc. The example .bashrc checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bashrc. | ** The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bashrc. The example .bashrc checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bashrc. | ||
** If you had a previously existing .bashrc, you should merge its content in .my_bashrc. Be careful not to needlessly duplicate content in your new .bashrc. | |||
** This is the file in which to put things like setting your prompt, defining aliases etc | ** This is the file in which to put things like setting your prompt, defining aliases etc | ||
We strongly recommend you '''not''' setup UPS packages (other than UPS itself) in your login scripts. Experience has shown that this will sometimes interfere with the correct operation of Mu2e software. In particular, if your login scripts source login/setup scripts from other experiments you should remove them from your login scripts. The recommended pattern is to keep your login scripts free of experiment specific code and to maintain separate per-experiment scripts that you source by hand after logging in; you should only source one of these scripts in any given shell. | |||
Line 26: | Line 48: | ||
# The Fermilab maintained systems execute /etc/profile.d/*.sh for login shells and /etc/bashrc for non-login shells. This may be different on other machines. | # The Fermilab maintained systems execute /etc/profile.d/*.sh for login shells and /etc/bashrc for non-login shells. This may be different on other machines. | ||
# We recommend that you '''not''' use ~/.login, ~/.profile, ~/.shrc or ~/.cshrc . There is no reason for them to be present in a Mu2e environment. | # We recommend that you '''not''' use ~/.login, ~/.profile, ~/.shrc or ~/.cshrc . There is no reason for them to be present in a Mu2e environment. | ||
===Expert Only=== | |||
If you choose that your login scripts will not setup UPS as described above, you will need to remember to do the following: everywhere that the Mu2e instructions tell you to | |||
> setup mu2e | |||
you should instead: | |||
> source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh | |||
== How Bash Startup Scripts Run == | == How Bash Startup Scripts Run == |
Revision as of 02:46, 4 October 2023
We use the bash linux shell exclusively for the interactive command line and shell scripting. In general, this is the most powerful shell and the professional's choice.
Setup scripts
These are also known as login scripts.
If you are a bash expert, read this and make your own decisions. See also #Expert Only, below.
If you are not a bash expert, follow the steps described here. When you first get a Mu2e computing account, we recommend that you copy two example files, .bash_login and .bashrc, to your home directory. The Mu2e computing environment, and the Mu2e documentation, presume that you setup UPS in your login script and these login scripts do that. You will see the use of the UPS system in the tutorials and we will describe it as you use it. The short version is that UPS is a system that makes additional software available in your working shell; you can choose to create different environments in different shell windows. If you have a lot of time you can read about UPS in advance but it's not necessary.
First, check to see if you have already have existing versions of these files:
> cd ~ > ls .bash_profile .bashrc .bash_profile .bashrc
If you have these files, make backup copies because the next steps will overwrite them:
> cd ~ > mv .bash_profile .bash_profile.sav # only if needed > mv .bashrc .bashrc.sav # only if needed > curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bash_profile > curl -O https://raw.githubusercontent.com/Mu2e/Bootstrap/main/dotFiles/.bashrc
In the above, the option argument to curl is a capital letter O, not the numeral zero. You can also click on the links to look at the files before your download them.
Here are some comments about each of these files.
- Example ~/.bash_profile
- This file is executed at the start of each login shell.
- It sources /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups.sh, which initializes the UPS system for use in your shell.
- The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bash_profile. The example .bash_profile checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bash_profile.
- If you had a previously existing .bash_profile, you should merge its content in .my_bash_profile. Be careful not to needlessly duplicate content in your new .bash_profile.
- It sources ~/.bashrc, which is not executed by the system for login shells.
- It defines BASH_ENV to point to ~/.bashrc so that is run for non-interactive shells
- Example ~/.bashrc
- This file is executed at the start of each non-login shells
- It sources /etc/bashrc for system settings
- The recommended practice is that you not modify this file. If you want to add personal content, create a file named ~/.my_bashrc. The example .bashrc checks if this file exists and, if it does, it sources it. The reason for this request is to simplify future upgrades to the recommended ~/.bashrc.
- If you had a previously existing .bashrc, you should merge its content in .my_bashrc. Be careful not to needlessly duplicate content in your new .bashrc.
- This is the file in which to put things like setting your prompt, defining aliases etc
We strongly recommend you not setup UPS packages (other than UPS itself) in your login scripts. Experience has shown that this will sometimes interfere with the correct operation of Mu2e software. In particular, if your login scripts source login/setup scripts from other experiments you should remove them from your login scripts. The recommended pattern is to keep your login scripts free of experiment specific code and to maintain separate per-experiment scripts that you source by hand after logging in; you should only source one of these scripts in any given shell.
Other notes:
- The Fermilab maintained systems execute /etc/profile.d/*.sh for login shells and /etc/bashrc for non-login shells. This may be different on other machines.
- We recommend that you not use ~/.login, ~/.profile, ~/.shrc or ~/.cshrc . There is no reason for them to be present in a Mu2e environment.
Expert Only
If you choose that your login scripts will not setup UPS as described above, you will need to remember to do the following: everywhere that the Mu2e instructions tell you to
> setup mu2e
you should instead:
> source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
How Bash Startup Scripts Run
This section describes which of the shell startup scripts are sourced in which circumstances. It is correct for bash on SL7.
- ssh hostname
- This creates an interactive login shell that sources the following startup scripts
- /etc/profile
- This sources all of the files matching /etc/profile.d/*.sh
- ~/.bash_profile, if present
- If ~/.bash_profile is not present then source ~/.profile, if present
- /etc/profile
- This creates an interactive login shell that sources the following startup scripts
- ssh hostname command
- This creates a non-login shell that sources the following scripts
- ~/.bashrc
- Whether the shell is interactive or non-interactive depends on the command. If the command is something like "xterm" that expects input from a terminal, it will be interactive; if it is something like "ls /path", which does not expect input from a terminal, it will be non-interactive.
- ~/.bashrc
- This creates a non-login shell that sources the following scripts
- When you create a bash subshell (which is done implicitly when you run most commands that are not shell built-ins), the subshell will source
- The file pointed to by $BASH_ENV, provided $BASH_ENV is a non-empty string.
- When you create a bash subshell run in POSIX compatibility mode ( ie executed as /bin/sh, not as /bin/bash ) the subshell will source
- The file pointed to by $ENV, provided $ENV is a non-empty string
The software deployed by Fermilab assumes that you have followed the recommendations below. If you do not, then some software installed by Fermilab may not work correctly.
- Your ~/.bashrc should source /etc/bashrc
- Your ~/.bash_profile should source your ~/.bashrc
Notes:
- On mu2egpvm*, /bin/sh is a symlink to /bin/bash
- On mu2egpvm*, /bin/slogin is a symlink to /bin/ssh
- a method to determine what files a login shell is opening:
echo exit | strace bash -li |& grep '^open' > login.txt
Standards
- scripts that intended to be executed by the average user at the command line, should be added to the user's path and can be named with no extension, so it looks built-in.
- scripts that are used in complex packages, or for special purposes, such as in monitoring, should have extension ".sh"
- scripts intended to be sourced should have the extension ".sh".
References
The Linux Documentation Project publishes a number of guides that are available online. We suggest:
- Bash Guide for Beginners, a comprehensive introduction.
- BASH Programming - Introduction HOW-TO, a shorter but less complete introduction.
- Advanced Bash Scripting Guide, starts at the beginning put goes more quickly to advanced ideas.
Beginner Cheat Sheet
Files and directories
cp <x> <y> % copy <x> to <y> mv <x> <y> % move <x> to <y> mkdir <x> % makes new directory <x> pwd %shows current directory cd <x> % goes into folder <x> rm <x> % remove file <x> ln -s <x> <y> % makes a soft link between real file <x> and local pointer <y> ls % lists documents in current directory cat <file> % list the whole file more <file> % types <file> in chunks, <space> goes to next chunk less <file> % similar to more head <file> % types the first N lines tail <file> % types the last N lines find * % list all of the files
Process management
CTL-Z % pause the current process and return to console bg % allow the process you just paused to run in background jobs % shows what you're running ps % shows processes <c> > file % output of command c goes to file <file> <c> >& file % errors from command c goes to file <file> <c> >> file % appends output of <c> onto end of <file> <c> >>& file % appends error of <c> onto end of <file>
Help
man <c> % help on command <c> <c> -h % sometimes has help this way as well
Editing and strings
sed s/<a>/<b>/ <file> > grep <a> <b> % print out all lines in <b> which contain string <a> sort <a> % sort the file a diff <a> <b> % print out the difference between <a> and <b> xemacs <a> % edit the file <a>
Environmentals
export <X>=<y> % will make $X refer to <y> export PATH=<y>:${PATH} %will append <y> to $PATH
important environmentals are:
$HOST % this computers's name $USER % your username $HOME % your home area $PWD % the current directory $PATH % where unix looks for code to execute $PYTHONPATH % where unix looks for python modules to import $LD_LIBRARY_PATH % where unix looks for shared libraries
You normally want to append to the PATH. Just setting them to <x> wipes out all the other stuff they were already set to.