LinuxFAQ

Introduction

The entries on this FAQ discuss issues related to linux, its shells and shell scripting languages.

On a unix machine, you open a terminal window into which you can type commands, the agent that figures out what you want to do and then gets the job done, is called a shell. There are many shells in common use, sh, bash, csh and tcsh to name the most commonly used. Instead of typing commands interactively one can collect many command into a file and ask the shell the execute the file as if you had typed the commands in by hand; this sort of file is called a shell-script.

On all of the Fermilab maintained Mu2e machines, the shell that you get when you login or open a new terminal window will be bash. Most of the scripts that drive Mu2e code are written in bash; in particluar all of the scripts that control grid jobs are written in bash.

What Bash References are Recommended?

The Linux Documentation Project publishes a number of guides that are available online. We suggest:

• Bash Guide for Beginners : A comprehensive introduction.
• BASH Programming - Introduction HOW-TO : A shorter but less complete introduction.
• Advanced Bash Scripting Guide : Starts at the beginning put goes more quickly to advanced ideas.

What Unix References are Recommended?

Most of the unix questions that you will have will really be questions about bash; for those questions see the FAQ entry about bash . For non-bash questions you can look at the various guides published by The Linux Documentation Project .

Fermilab uses Linux, which (as of version 7) is very similar to CentOS 7, so for answers to generic system questions one may search the internet for CentOS tips. Fermilab-specific configuration is described [1]. If you are configuring a private or university laptop you may want to install only a subset of fermilab context packages and skip e.g. the fermilab-conf_doe-banner-login-screen.

Bash Startup Scripts

This section describes which of the shell startup scripts are sourced in which circumstances. It is correct for bash on SL7.

1. ssh hostname
   • This creates an interactive login shell that sources the following startup scripts
     • /etc/profile
       • This sources all of the files matching /etc/profile.d/*.sh
     • ~/.bash_profile, if present
     • If ~/.bash_profile is not present then source ~/.profile, if present
2. ssh hostname command
   • This creates a non-login shell that sources the following scripts
     • ~/.bashrc
       • Whether the shell is interactive or non-interactive depends on the command. If the command is something like "xterm" that expects input from a terminal, it will be interactive; if it is something like "ls /path", which does not expect input from a terminal, it will be non-interactive.
3. When you create a bash subshell (which is done implicitly when you run most commands that are not shell built-ins), the subshell will source
   • The file pointed to by $BASH_ENV, provided $BASH_ENV is a non-empty string.
4. When you create a bash subshell run in POSIX compatibility mode ( ie executed as /bin/sh, not as /bin/bash ) the subshell will source
   • The file pointed to by $ENV, provided $ENV is a non-empty string

The software deployed by Fermilab assumes that you have followed the recommendations below. If you do not, then some software installed by Fermilab may not work correctly.

1. Your ~/.bashrc should source /etc/bashrc
2. Your ~/.bash_profile should source your ~/.bashrc

Notes:

1. On mu2egpvm*, /bin/sh is a symlink to /bin/bash
2. On mu2egpvm*, /bin/slogin is a symlink to /bin/sh

Extra repositories

If one needs a system software package ("RPM") that is not available in the default SL7 repositories, additional software sources can be enabled. Per SL7 notes

   yum install yum-conf-repos

should be done first to obtain a list of additional repositories. Then a command like

  yum install yum-conf-epel

for example will enable the EPEL repository. A list of all repositories configured for your system can be found in

  ls /etc/yum.repos.d/*.repo

files.

Study Questions about Shells

When you learn about shells and shell scripting there is a lot of information available. If you can answer all of the following quesitons you will have a solid grounding in the basics:

1. What is a shell?
2. What are sh, bash, csh, tcsh, ksh, zsh?
3. What is a shell script
4. What is a subshell?
5. You can either source a shell script or execute it.
   • What is the difference in syntax?
   • What is the difference in behaviour?
   • What is the meanging of the #! line at the start of a shell script? What happens if it is not there?
   • When you source a shell script, which shell does it run under?
   • When you execute a shell script, which shell does it run under?
6. In some shells, "." is a synonym for source. Which shells?
7. What is the difference between a shell variable and an environment variable?
   • Difference is syntax for defining them?
   • Difference in scope?
   • There is a convention that one type of variable is written in all capital letters and another in all lower case letters. Which is which?
8. What environment does a subshell inherited from its parent?
9. If you change the environment in a subshell, does this change the environment of its parent?
10. If I start one subshell and later change my environment, does this affect the environment of the subshell?
11. When you dereference a variable ( shell or environment ) sometimes the syntax is $var and other times it is ${var}. Is this 100% optional or are the {} sometimes required?
12. What is an alias and how to does it work?
13. When you type a command the shell can resolve the command from different places:
    • In the path
    • An alias
    • A shell built-it
    • A shell function.

        What is the order of precedence for checking these 4 places to find a command?
    

14. How can you ask a shell to tell you where it found a command?
15. Which directory is referred to by ~? By ~username? By ~/username?
16. Inside C++ code or ROOT code, when I try to open a file using the ~ syntax it does not work. Why?
17. What entity processes wildcards?
18. What is the difference between quoting things with " vs '? (Double vs single quotes).
19. What does it mean to escape a special character? What is the syntax to do so?
20. If I enclose a string in "back ticks" ( the ` character) what happens?
21. What happens if you end a line with \ (the backslash is the last character on the line) ?
22. What happens if the last visible character on a line is \ but there is whitespace after it?
23. What is LD_LIBRARY_PATH?
24. How do you pass arguments to a script?
25. Inside a script, how do you access its arguments?
26. What are standard in, standard error and standard out ( also known as stderr and stdout?)?
27. How do you redirect stdout? stderr?
28. How do you redirect both stdout and stderr to different places?
29. How can I redirect stdin to a file?
30. How can I redirect stdin to be inline the current script?
31. What are pipes and how are they used?
32. How can you concatenate many commands together onto one line?
33. When is .bash_profile run? Is it sourced or executed?
34. When is .bashrc run? Is it sourced or executed?
35. What sort of things belong on .bash_profile and which in .bashrc?
36. If you do not have a .bash_profile and/or a .bashrc, what will bash look for instead?
37. What is the your "current working directory"? How do you learn what it is?
38. What are your uid and gid? How do you learn their values?
39. Which file protection groups do you belong to?
40. What is the newgrp command and when should you used it?
41. How do you tell who is allowed to read/write/execute a file?
42. How do you change who is allowed to read/write/execute a file?
43. Both binary files and ascii files can be executable? What does it mean for a file with text in it to be executable?
44. What is a umask? How do you find its value? How do you change its value?
45. What does the sticky bit in a directory umask mean?
46. In bash, when you should use return and when should you use exit?
47. Why is it a bad idea to use arguments with bash scripts that are designed to be sourced?
48. How can you send the result of you work to both the screen and a file?
49. Some Mu2e specific questions:
    • Why must you source, not execute the mu2e setup scripts?
    • What are FHICL_FILE_PATH, MU2E_SEARCH_PATH?
    • Why are we using bash, not tcsh?
    • Our grid scripts use trap - what is it and why do we use it?