Difference between revisions of "CodeTools"

From Mu2eWiki
Jump to navigation Jump to search
 
(32 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
=Introduction=
 
=Introduction=
A couple of 'clang' tools are now available for the standardisation of code style and to sniff out bug-prone behaviour, clang-format and clang-tidy.
+
A couple of 'clang' tools are now available for the standardisation of code style and to sniff out bug-prone behaviour: clang-format and clang-tidy.
  
To use these tools on SL7 (with CVMFS), please set them up like this:
+
To use these tools on SLF7, please set them up like this:
 
<pre>
 
<pre>
source /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups
+
mu2einit          #  Or: source /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups
setup mu2e
+
setup clang v14_0_6c
setup clang v5_0_1
 
 
</pre>
 
</pre>
  
Line 29: Line 28:
  
 
</pre>
 
</pre>
 
'''Please note that as of the current HEAD of Mu2e/Offline, there are no .clang-format or .clang-tidy configuration files.'''
 
  
 
=clang-format=
 
=clang-format=
 
Clang-format re-formats code according to a configuration file. Having a consistent code style and format yields numerous benefits:
 
Clang-format re-formats code according to a configuration file. Having a consistent code style and format yields numerous benefits:
* Cleaner diffs
+
* Consistent formatting between revisions means cleaner diffs
* Code is much easier to read
+
* Consistently formatted code is on the whole easier to read
* Pesky trailing whitespace is removed
+
* Whitespace problems are ironed out (e.g. no trailing whitespace)
  
 
== Basic Usage ==
 
== Basic Usage ==
Line 50: Line 47:
 
git diff
 
git diff
 
</pre>
 
</pre>
 +
 +
Setting up the entire clang product may introduce conflicts in the libraries used in gcc in the same process.  To avoid setting the product, format can be used standalone:
 +
alias format=/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/bin/clang-format
  
 
== IDE Integration ==
 
== IDE Integration ==
Clang-format is widely supported:
+
Clang-format is widely supported (see also [[Editors]]):
 
* emacs: [https://clang.llvm.org/docs/ClangFormat.html#emacs-integration]
 
* emacs: [https://clang.llvm.org/docs/ClangFormat.html#emacs-integration]
 
* vim: [https://clang.llvm.org/docs/ClangFormat.html#vim-integration]
 
* vim: [https://clang.llvm.org/docs/ClangFormat.html#vim-integration]
 
* vscode: [https://code.visualstudio.com/docs/cpp/cpp-ide#_code-formatting]
 
* vscode: [https://code.visualstudio.com/docs/cpp/cpp-ide#_code-formatting]
 +
 +
===emacs===
 +
 +
Here is one way to incorporate format in emacs without setting up clang, which may introduce conflicts with gcc.
 +
(add-to-list 'exec-path "/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/bin")
 +
(load "/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/share/clang/clang-format.el")
 +
(global-set-key "\C-\M-f" 'clang-format-buffer )
 +
 +
 +
===XCode===
 +
Xcode: You can create a 'Text Service' and run clang-format from the shell script. See [https://developer.apple.com/library/archive/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/MakeaSystem-WideService.html]
 +
 +
<pre>
 +
# Shell script for Xcode/apple 'Text Service'
 +
# Input (is entire selection +) receives all selected text
 +
# Select: output replaces selected text
 +
# You can select text in Xcode and run this 'Text Service' to format the selected text and replace it with formatted output.
 +
 +
/usr/bin/local/clang-format # replace /usr/bin/local with the install location of clang-format
 +
</pre>
  
 
=clang-tidy=
 
=clang-tidy=
Line 62: Line 82:
 
* Enforce C++ core guidelines
 
* Enforce C++ core guidelines
 
* Sniff out bug-prone code
 
* Sniff out bug-prone code
 +
* Performance tips e.g. passing by const& instead of value.
 +
 +
What clang-tidy looks for will depend on the enabled checks. A list of available checks can be found at [https://releases.llvm.org/14.0.0/tools/clang/tools/extra/docs/clang-tidy/checks/list.html v14 checks]
 +
 +
== IDE Integration ==
 +
Clang-tidy output is best served by an IDE or Text Editor.
 +
 +
Most editors will have support for clang-tidy and display warning messages as you update code.
  
What clang-tidy looks for will depend on the enabled checks. A list of available checks can be found at [https://clang.llvm.org/extra/clang-tidy/checks/list.html]
+
* Clangd is a widely supported 'language server' interface to clang tools. Please see [https://clang.llvm.org/extra/clangd/] for details and [https://clang.llvm.org/extra/clangd/Installation.html] for installation instructions. It works especially well for VS Code.
 +
* XCode Clang-Analyzer [https://clang-analyzer.llvm.org/xcode.html]
  
== Basic Usage ==
+
To get this to work, you will need to copy over an up-to-date version of compile_commands.json for your editor to correctly run clang-tidy. If your editor environment is not SLF7, you may need to generate this with the above steps on a SLF7 machine (or docker container) that can set up and compile the Offline software. Make sure that when you move it to a different machine that the "directory" value on all JSON list entries point to the correct location. You can fix this quickly via a search-and-replace all operation.
Clang-tidy requires a compile_commands.json to run correctly, as it needs to know the compile flags. This can be generated by a script which scrapes these from SCons output [https://gist.github.com/ryuwd/47418eacdadf5369ab4e99492d583f19]
 
  
To analyse '''<files>''', and apply properly-formatted fixes ''in place'':
+
In the editor environment you copy this to, you will need the correct CVMFS mounts (mu2e.opensciencegrid.org, fermilab.opensciencegrid.org) for clang-tidy to access the external headers.
<pre>
 
cd <Offline directory>
 
curl https://gist.githubusercontent.com/ryuwd/47418eacdadf5369ab4e99492d583f19/raw/e7e0c4559f2af3196c665b0ab9c3aa2bd8878215/generate_compile_commands.py > generate_compile_commands.py
 
setup mu2e
 
source setup.sh
 
python generate_compile_commands.py
 
  
# only works for SL7 (CVMFS product)
+
== Command Line Usage ==
# other systems: set CLANG_FQ_DIR to where clang include, share, and bin directories are installed
 
CLANG_TIDY_ARGS="-extra-arg=-isystem$CLANG_FQ_DIR/include/c++/v1 -p . -fix -format"
 
CLANG_TIDY_RUNNER="${CLANG_FQ_DIR}/share/clang/run-clang-tidy.py"
 
  
${CLANG_TIDY_RUNNER} ${CLANG_TIDY_ARGS} <files>
+
===AL9===
  
 +
<pre>
 +
mu2einit
 +
spack_load_current llvm  # can be abbreviated: slc llvm
 +
muse setup
 +
muse build --mu2eCBD ...
 +
run-clang-tidy -p $MUSE_BUILD_DIR  file1.cc file2.cc ...
 +
</pre>
 +
 +
Or, to do all files in the build:
 +
<pre>
 +
run-clang-tidy -p $MUSE_BUILD_DIR
 
</pre>
 
</pre>
  
As you can probably see, the set-up required here is non-trivial, requiring the use of an external script. This may motivate adding Compilation DB support into SConstruct.
 
  
'''N.B. Not all checks will provide fixes. Some warning messages will require manual review.'''
+
====Notes====
 +
# The <code>spack_load_current</code> command must come after <code>mu2einit</code> and before <code>muse setup</code>. It finds the currently recommended version of the spack llvm package and loads it. After <code>muse setup</code> it is no longer possible to <code>spack load</code> a package.
 +
# codetools is still coming from UPS.  It is script-only and so does not need a spack build.  At a later date it will be moved to spack.
 +
# <code>muse build --mu2eCBD</code> does the build and creates the file $MUSE_BUILD_DIR/compile_commands.json.  This is the compilation database and contains a record of the command needed to compile each .cc file.  This is used by clang-tidy to know what options it should use.  It is a text file that you can read.
 +
#*  If you do <code>muse build</code> and later come back to do <code>muse bulid --mu2eCBD</code> the compilation DB will be empty since nothing was actually built.  The solution is to remove the existing build ( <code>muse build -c</code> ) and rebuild using <code>--mu2eCBD</code>.
 +
# In SL7, the UPS package was called clang but in AL9 the spack package is llvm.  To learn about clang vs llvm, see [https://stackoverflow.com/questions/5708610/llvm-vs-clang-on-os-x] and [https://alibabatech.medium.com/gcc-vs-clang-llvm-an-in-depth-comparison-of-c-c-compilers-899ede2be378#:~:text=LLVM%20can%20serve%20as%20a,superior%20to%20that%20of%20GCC.]
 +
# clang-tidy has mechanism which will apply fixes to code automatically, which is not turned on here.
 +
 
 +
===SL7===
 +
<pre>
 +
cd Muse workDir
 +
mu2einit
 +
muse setup
 +
# build, including the compilation db
 +
muse build --mu2eCBD ...
  
== IDE Integration ==
+
setup codetools
Some editors will have support for clang-tidy and display warning messages as you update code.
+
setup clang v14_0_6c
  
To get this to work, you will need to copy over an up-to-date version of compile_commands.json for your editor to correctly run clang-tidy. If your editor environment is not SLF7, you may need to generate this with the above steps on a SLF7 machine that can set up and compile the Offline software. You do not need to fully compile Offline to generate a compile_commands.json.
+
mu2e-clang-tidy <files>
  
In the editor environment you copy this to, you will need the correct CVMFS mounts (mu2e.opensciencegrid.org, fermilab.opensciencegrid.org) for clang-tidy to access the external headers.
+
</pre>
 +
clang-tidy has mechanism which will apply fixes to code automatically, which is not turned on here.
  
 
=Include What You Use (IWYU)=
 
=Include What You Use (IWYU)=
 +
 +
This [https://include-what-you-use.org/ utility] will examine the include files in a piece of code and remove the ones that aren't needed and include the file defining the classes that are used.  It requires the scons compilation database, which can be built:
 +
muse build --mu2eCBD
 +
setup iwyu
 +
the compilation database will be in $MUSE_BUILD_BASE
 +
 +
A typical iwyu command is
 +
iwyu_tool.py -p $MUSE_BUILD_BASE/compile_commands.json Mu2eUtilities/src/*.cc -j 6 | fix_includes.py --comments --noreorder
 +
Instructions are at the [https://include-what-you-use.org/ website] and the [https://github.com/include-what-you-use/include-what-you-use/blob/master/README.md repo]

Latest revision as of 02:46, 21 August 2024

Introduction

A couple of 'clang' tools are now available for the standardisation of code style and to sniff out bug-prone behaviour: clang-format and clang-tidy.

To use these tools on SLF7, please set them up like this:

mu2einit           #  Or: source /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups
setup clang v14_0_6c

clang and its tools can be installed on many systems:

# Debian, Ubuntu
sudo apt-get install clang-format clang-tidy

# MacOS (Homebrew)
brew install clang-format # smaller package, just clang-format
# or:
# includes clang-format and clang-tidy - but takes up a lot more space
brew install llvm 
ln -s "$(brew --prefix llvm)/bin/clang-format" "/usr/local/bin/clang-format"
ln -s "$(brew --prefix llvm)/bin/clang-tidy" "/usr/local/bin/clang-tidy"

# Arch Linux
sudo pacman -S clang

# Fedora
sudo dnf install clang

clang-format

Clang-format re-formats code according to a configuration file. Having a consistent code style and format yields numerous benefits:

  • Consistent formatting between revisions means cleaner diffs
  • Consistently formatted code is on the whole easier to read
  • Whitespace problems are ironed out (e.g. no trailing whitespace)

Basic Usage

To re-format a file in place:

clang-format -i <file(s)>

Glob patterns are also supported e.g. Analyses/src/*.cc

To see what it changed:

git diff

Setting up the entire clang product may introduce conflicts in the libraries used in gcc in the same process. To avoid setting the product, format can be used standalone:

alias format=/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/bin/clang-format

IDE Integration

Clang-format is widely supported (see also Editors):

emacs

Here is one way to incorporate format in emacs without setting up clang, which may introduce conflicts with gcc.

(add-to-list 'exec-path "/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/bin")
(load "/cvmfs/mu2e.opensciencegrid.org/artexternals/clang/v5_0_1/Linux64bit+3.10-2.17/share/clang/clang-format.el")
(global-set-key "\C-\M-f" 'clang-format-buffer )


XCode

Xcode: You can create a 'Text Service' and run clang-format from the shell script. See [4]

# Shell script for Xcode/apple 'Text Service'
# Input (is entire selection +) receives all selected text
# Select: output replaces selected text
# You can select text in Xcode and run this 'Text Service' to format the selected text and replace it with formatted output.

/usr/bin/local/clang-format # replace /usr/bin/local with the install location of clang-format

clang-tidy

Clang-tidy is a static code analyzer which can perform a number of checks, including but not limited to:

  • Enforce variable naming conventions
  • Enforce C++ core guidelines
  • Sniff out bug-prone code
  • Performance tips e.g. passing by const& instead of value.

What clang-tidy looks for will depend on the enabled checks. A list of available checks can be found at v14 checks

IDE Integration

Clang-tidy output is best served by an IDE or Text Editor.

Most editors will have support for clang-tidy and display warning messages as you update code.

  • Clangd is a widely supported 'language server' interface to clang tools. Please see [5] for details and [6] for installation instructions. It works especially well for VS Code.
  • XCode Clang-Analyzer [7]

To get this to work, you will need to copy over an up-to-date version of compile_commands.json for your editor to correctly run clang-tidy. If your editor environment is not SLF7, you may need to generate this with the above steps on a SLF7 machine (or docker container) that can set up and compile the Offline software. Make sure that when you move it to a different machine that the "directory" value on all JSON list entries point to the correct location. You can fix this quickly via a search-and-replace all operation.

In the editor environment you copy this to, you will need the correct CVMFS mounts (mu2e.opensciencegrid.org, fermilab.opensciencegrid.org) for clang-tidy to access the external headers.

Command Line Usage

AL9

mu2einit
spack_load_current llvm  # can be abbreviated: slc llvm
muse setup
muse build --mu2eCBD ...
run-clang-tidy -p $MUSE_BUILD_DIR  file1.cc file2.cc ...

Or, to do all files in the build:

run-clang-tidy -p $MUSE_BUILD_DIR


Notes

  1. The spack_load_current command must come after mu2einit and before muse setup. It finds the currently recommended version of the spack llvm package and loads it. After muse setup it is no longer possible to spack load a package.
  2. codetools is still coming from UPS. It is script-only and so does not need a spack build. At a later date it will be moved to spack.
  3. muse build --mu2eCBD does the build and creates the file $MUSE_BUILD_DIR/compile_commands.json. This is the compilation database and contains a record of the command needed to compile each .cc file. This is used by clang-tidy to know what options it should use. It is a text file that you can read.
    • If you do muse build and later come back to do muse bulid --mu2eCBD the compilation DB will be empty since nothing was actually built. The solution is to remove the existing build ( muse build -c ) and rebuild using --mu2eCBD.
  4. In SL7, the UPS package was called clang but in AL9 the spack package is llvm. To learn about clang vs llvm, see [8] and [9]
  5. clang-tidy has mechanism which will apply fixes to code automatically, which is not turned on here.

SL7

cd Muse workDir
mu2einit
muse setup
# build, including the compilation db
muse build --mu2eCBD ...

setup codetools
setup clang v14_0_6c

mu2e-clang-tidy <files>

clang-tidy has mechanism which will apply fixes to code automatically, which is not turned on here.

Include What You Use (IWYU)

This utility will examine the include files in a piece of code and remove the ones that aren't needed and include the file defining the classes that are used. It requires the scons compilation database, which can be built:

muse build --mu2eCBD
setup iwyu

the compilation database will be in $MUSE_BUILD_BASE

A typical iwyu command is

iwyu_tool.py -p $MUSE_BUILD_BASE/compile_commands.json Mu2eUtilities/src/*.cc -j 6 | fix_includes.py --comments --noreorder

Instructions are at the website and the repo