Skip to content
Snippets Groups Projects
Commit f1c0c416 authored by Jan Frenzel's avatar Jan Frenzel
Browse files

Added a specific file list containing all files to skip for each 

pattern. This resolves #219.
parent b1b3fe12
No related branches found
No related tags found
2 merge requests!415Added a specific file list containing all files to skip for each,!410Added a specific file list containing all files to skip for each pattern.
Hide whitespace changes
Inline Side-by-side
doc.zih.tu-dresden.de/util/grep-forbidden-patterns.sh
+ 33
− 21
View file @ f1c0c416
...@@ -6,41 +6,50 @@ scriptpath=${BASH_SOURCE[0]} ...@@ -6,41 +6,50 @@ scriptpath=${BASH_SOURCE[0]}
basedir=`dirname "$scriptpath"` basedir=`dirname "$scriptpath"`
basedir=`dirname "$basedir"` basedir=`dirname "$basedir"`
#This is the ruleset. Each line represents a rule of tab-separated fields. #This is the ruleset. Each rule consists of a message (first line), a tab-separated list of files to skip (second line) and a pattern specification (third line).
#A pattern specification is a tab-separated list of fields:
#The first field represents whether the match should be case-sensitive (s) or insensitive (i). #The first field represents whether the match should be case-sensitive (s) or insensitive (i).
#The second field represents the pattern that should not be contained in any file that is checked. #The second field represents the pattern that should not be contained in any file that is checked.
#Further fields represent patterns with exceptions. #Further fields represent patterns with exceptions.
#For example, the first rule says: #For example, the first rule says:
# The pattern \<io\> should not be present in any file (case-insensitive match), except when it appears as ".io". # The pattern \<io\> should not be present in any file (case-insensitive match), except when it appears as ".io".
ruleset="The word \"IO\" should not be used, use \"I/O\" instead. ruleset="The word \"IO\" should not be used, use \"I/O\" instead.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i \<io\> \.io i \<io\> \.io
\"SLURM\" (only capital letters) should not be used, use \"Slurm\" instead. \"SLURM\" (only capital letters) should not be used, use \"Slurm\" instead.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
s \<SLURM\> s \<SLURM\>
\"File system\" should be written as \"filesystem\", except when used as part of a proper name. \"File system\" should be written as \"filesystem\", except when used as part of a proper name.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i file \+system HDFS i file \+system HDFS
Use \"ZIH systems\" or \"ZIH system\" instead of \"Taurus\". \"taurus\" is only allowed when used in ssh commands and other very specific situations. Use \"ZIH systems\" or \"ZIH system\" instead of \"Taurus\". \"taurus\" is only allowed when used in ssh commands and other very specific situations.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i \<taurus\> taurus\.hrsk /taurus /TAURUS ssh ^[0-9]\+:Host taurus$ i \<taurus\> taurus\.hrsk /taurus /TAURUS ssh ^[0-9]\+:Host taurus$
\"HRSKII\" should be avoided, use \"ZIH system\" instead. \"HRSKII\" should be avoided, use \"ZIH system\" instead.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i \<hrskii\> i \<hrskii\>
The term \"HPC-DA\" should be avoided. Depending on the situation, use \"data analytics\" or similar. The term \"HPC-DA\" should be avoided. Depending on the situation, use \"data analytics\" or similar.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i hpc[ -]\+da\> i hpc[ -]\+da\>
\"ATTACHURL\" was a keyword in the old wiki, don't use it. \"ATTACHURL\" was a keyword in the old wiki, don't use it.
doc.zih.tu-dresden.de/README.md
i attachurl i attachurl
Replace \"todo\" with real content. Replace \"todo\" with real content.
doc.zih.tu-dresden.de/README.md
i \<todo\> <!--.*todo.*--> i \<todo\> <!--.*todo.*-->
Avoid spaces at end of lines. Avoid spaces at end of lines.
doc.zih.tu-dresden.de/README.md
i [[:space:]]$ i [[:space:]]$
When referencing partitions, put keyword \"partition\" in front of partition name, e. g. \"partition ml\", not \"ml partition\". When referencing partitions, put keyword \"partition\" in front of partition name, e. g. \"partition ml\", not \"ml partition\".
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i \(alpha\|ml\|haswell\|romeo\|gpu\|smp\|julia\|hpdlf\|scs5\)-\?\(interactive\)\?[^a-z]*partition i \(alpha\|ml\|haswell\|romeo\|gpu\|smp\|julia\|hpdlf\|scs5\)-\?\(interactive\)\?[^a-z]*partition
Give hints in the link text. Words such as \"here\" or \"this link\" are meaningless. Give hints in the link text. Words such as \"here\" or \"this link\" are meaningless.
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i \[\s\?\(documentation\|here\|this \(link\|page\|subsection\)\|slides\?\|manpage\)\s\?\] i \[\s\?\(documentation\|here\|this \(link\|page\|subsection\)\|slides\?\|manpage\)\s\?\]
Use \"workspace\" instead of \"work space\" or \"work-space\". Use \"workspace\" instead of \"work space\" or \"work-space\".
doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md
i work[ -]\+space" i work[ -]\+space"
# Whitelisted files will be ignored
# Whitespace separated list with full path
whitelist=(doc.zih.tu-dresden.de/README.md doc.zih.tu-dresden.de/docs/contrib/content_rules.md)
function grepExceptions () { function grepExceptions () {
if [ $# -gt 0 ]; then if [ $# -gt 0 ]; then
firstPattern=$1 firstPattern=$1
...@@ -55,22 +64,29 @@ function checkFile(){ ...@@ -55,22 +64,29 @@ function checkFile(){
f=$1 f=$1
echo "Check wording in file $f" echo "Check wording in file $f"
while read message; do while read message; do
IFS=$'\t' read -r -a files_to_skip
skipping=""
if (printf '%s\n' "${files_to_skip[@]}" | grep -xq $f); then
skipping=" -- skipping"
fi
IFS=$'\t' read -r flags pattern exceptionPatterns IFS=$'\t' read -r flags pattern exceptionPatterns
while IFS=$'\t' read -r -a exceptionPatternsArray; do while IFS=$'\t' read -r -a exceptionPatternsArray; do
if [ $silent = false ]; then if [ $silent = false ]; then
echo " Pattern: $pattern" echo " Pattern: $pattern$skipping"
fi fi
grepflag= if [ -z "$skipping" ]; then
case "$flags" in grepflag=
"i") case "$flags" in
grepflag=-i "i")
;; grepflag=-i
esac ;;
if grep -n $grepflag $color "$pattern" "$f" | grepExceptions "${exceptionPatternsArray[@]}" ; then esac
number_of_matches=`grep -n $grepflag $color "$pattern" "$f" | grepExceptions "${exceptionPatternsArray[@]}" | wc -l` if grep -n $grepflag $color "$pattern" "$f" | grepExceptions "${exceptionPatternsArray[@]}" ; then
((cnt=cnt+$number_of_matches)) number_of_matches=`grep -n $grepflag $color "$pattern" "$f" | grepExceptions "${exceptionPatternsArray[@]}" | wc -l`
if [ $silent = false ]; then ((cnt=cnt+$number_of_matches))
echo " $message" if [ $silent = false ]; then
echo " $message"
fi
fi fi
fi fi
done <<< $exceptionPatterns done <<< $exceptionPatterns
...@@ -138,10 +154,6 @@ if [[ ! -z $file ]]; then ...@@ -138,10 +154,6 @@ if [[ ! -z $file ]]; then
else else
for f in $files; do for f in $files; do
if [ "${f: -3}" == ".md" -a -f "$f" ]; then if [ "${f: -3}" == ".md" -a -f "$f" ]; then
if (printf '%s\n' "${whitelist[@]}" | grep -xq $f); then
echo "Skip whitelisted file $f"
continue
fi
checkFile $f checkFile $f
fi fi
done done
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
Nutzungsbedingungen | Datenschutzerklärung | Barrierefreiheitserklärung