Skip to content
Snippets Groups Projects
Commit 1972d6a1 authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Refactor script check-links 

- Merge functionality of two check links scripts into one
- Document usage of the script when working with docker container
parent 242025a5
No related branches found
No related tags found
2 merge requests!1138Automated merge from preview to main,!1129Refactor script check-links
Hide whitespace changes
Inline Side-by-side
doc.zih.tu-dresden.de/docs/contrib/contribute_container.md
+ 19
− 7
View file @ 1972d6a1
...@@ -205,22 +205,34 @@ wikiscript doc.zih.tu-dresden.de/util/check-no-floating.sh doc.zih.tu-dresden.de ...@@ -205,22 +205,34 @@ wikiscript doc.zih.tu-dresden.de/util/check-no-floating.sh doc.zih.tu-dresden.de
No one likes dead links. No one likes dead links.
Therefore, we check the internal and external links within the markdown source files. To check a Therefore, we check the internal and external links within the markdown source files.
single file, e.g. `doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md`, use: The script `doc.zih.tu-dresden.de/util/check-links.sh` allows to check a [single file](#single-file),
all [all git-modified files](#all-git-modified-files) and
[all markdown files of the compendium](#all-files). Please follow the instructions in the
correspoding subsection.
##### Single File
To check the links within a single file, e.g.
`doc.zih.tu-dresden.de/docs/software/big_data_frameworks.md`, use:
```bash ```bash
wikiscript doc.zih.tu-dresden.de/util/check-links.sh docs/software/big_data_frameworks.md wikiscript doc.zih.tu-dresden.de/util/check-links.sh docs/software/big_data_frameworks.md
``` ```
The script can also check all modified files, i.e., markdown files which are part of the repository ##### All Git-Modified Files
and different to the `main` branch. Use this script before committing your changes to make sure
your commit passes the CI/CD pipeline: The script can also check the links in all git-modified files, i.e., markdown files which are part
of the repository and different to the `preview` branch. Use this script before committing your
changes to make sure your commit passes the CI/CD pipeline.
```bash ```bash
wikiscript doc.zih.tu-dresden.de/util/check-links.sh wikiscript doc.zih.tu-dresden.de/util/check-links.sh -c
``` ```
To check all markdown file, which may take a while and give a lot of output, use: ##### All Files
Checking the links of all markdown files takes a moment of time:
```bash ```bash
wikiscript doc.zih.tu-dresden.de/util/check-links.sh -a wikiscript doc.zih.tu-dresden.de/util/check-links.sh -a
......
doc.zih.tu-dresden.de/util/check-links-non-mkdocs.sh deleted 100755 → 0
+ 0
− 103
View file @ 242025a5
#!/bin/bash
## Purpose:
## Checks internal links for all (git-)changed markdown files (.md) of the repository.
## We use the markdown-link-check script from https://github.com/tcort/markdown-link-check,
## which can either be installed as local or global module
## nmp npm install [--save-dev|-g] markdown-link-check
## module.
##
## Author: Martin.Schroschk@tu-dresden.de
set -eo pipefail
scriptpath=${BASH_SOURCE[0]}
basedir=`dirname "${scriptpath}"`
basedir=`dirname "${basedir}"`
usage() {
cat <<-EOF
usage: $0 [file | -a]
If file is given, checks whether all links in it are reachable.
If parameter -a (or --all) is given instead of the file, checks all markdown files.
Otherwise, checks whether any changed file contains broken links.
EOF
}
mlc=markdown-link-check
if ! command -v ${mlc} &> /dev/null; then
echo "INFO: ${mlc} not found in PATH (global module)"
mlc=./node_modules/markdown-link-check/${mlc}
if ! command -v ${mlc} &> /dev/null; then
echo "INFO: ${mlc} not found (local module)"
exit 1
fi
fi
echo "mlc: ${mlc}"
LINK_CHECK_CONFIG="${basedir}/util/link-check-config.json"
if [[ ! -f "${LINK_CHECK_CONFIG}" ]]; then
echo ${LINK_CHECK_CONFIG} does not exist
exit 1
fi
branch="preview"
if [[ -n "${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}" ]]; then
branch="origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}"
fi
function check_single_file(){
theFile="$1"
if [[ -e "${theFile}" ]]; then
echo "Checking links in ${theFile}"
if ! ${mlc} -q -c "${LINK_CHECK_CONFIG}" -p "${theFile}"; then
return 1
fi
fi
return 0
}
function check_files(){
any_fails=false
echo "Check files:"
echo "${files}"
echo ""
for f in ${files}; do
if ! check_single_file "${f}"; then
any_fails=true
fi
done
if [[ "${any_fails}" == true ]]; then
exit 1
fi
}
function check_all_files(){
files=$(git ls-tree --full-tree -r --name-only HEAD ${basedir}/ | grep '.md$' || true)
check_files
}
function check_changed_files(){
files=$(git diff --name-only "$(git merge-base HEAD "${branch}")" | grep '.md$' || true)
check_files
}
if [[ $# -eq 1 ]]; then
case $1 in
help | -help | --help)
usage
exit
;;
-a | --all)
check_all_files
;;
*)
checkSingleFile "$1"
;;
esac
elif [[ $# -eq 0 ]]; then
check_changed_files
else
usage
fi
doc.zih.tu-dresden.de/util/check-links.sh
+ 114
− 30
View file @ 1972d6a1
#!/bin/bash #!/bin/bash
# Purpose: # Purpose:
# Checks internal and external links for all markdown files (.md) of the repository. # Checks internal and external links for markdown files (.md) of the repository
# For link checking, we use html-proofer-plugin, which is configured in mkdocs.yml. # (i.e. HPC compendium). This script can check the URLs
# 1. in all markdown files
# 2. git-changed markdown files
# 3. single markdown file
#
# For option 1, we make use of the html-proofer-plugin, which is configured in mkdocs.yml.
# By default the link checker only inspects internal links. It will inspect internal and external # By default the link checker only inspects internal links. It will inspect internal and external
# links, when the env. variable ENABLED_HTMLPROOFER_EXTERNAL_URLS is set to true. # links, since we set the env. variable ENABLED_HTMLPROOFER_EXTERNAL_URLS to true in this script.
#
# For option 2 and 3, we use the markdown-link-check script from
# https://github.com/tcort/markdown-link-check, which can either be installed as local or global
# module (nmp npm install [--save-dev|-g] markdown-link-check).
#
# General remarks for future me:
# - html-proofer-plugin is heavily customizable (ignore certain urls or pages, etc.)
# - html-proofer-plugin is quite slow, since it requires to build the whole compendium
# - html-proofer-plugin works on rendered html files
# - html-proofer-plugin has no option to check a single page
#
# Author: Martin.Schroschk@tu-dresden.de
set -eo pipefail set -eo pipefail
export ENABLED_HTMLPROOFER=true scriptpath=${BASH_SOURCE[0]}
basedir=`dirname "${scriptpath}"`
basedir=`dirname "${basedir}"`
function usage(){ usage() {
cat << EOF cat <<-EOF
$0 [options] usage: $0 [file | -a | -c]
Check links in documentation.
Options: Options:
-a Check internal and external links (default: internal links only) <file> Check links in the provided markdown file
-h Show help message -a; --all Check links in all markdown files
-c; --changed Check links in all git-changed files
EOF EOF
} }
while getopts ":a" option; do mlc=markdown-link-check
case ${option} in if ! command -v ${mlc} &> /dev/null; then
a) echo "INFO: ${mlc} not found in PATH (global module)"
export ENABLED_HTMLPROOFER_EXTERNAL_URLS=true mlc=./node_modules/markdown-link-check/${mlc}
echo "Info: Checking internal and external links. Might take some time." if ! command -v ${mlc} &> /dev/null; then
;; echo "INFO: ${mlc} not found (local module)"
h) exit 1
usage fi
exit;; fi
\?)
echo "Error: Invalid option." echo "mlc: ${mlc}"
usage
exit;; LINK_CHECK_CONFIG="${basedir}/util/link-check-config.json"
esac if [[ ! -f "${LINK_CHECK_CONFIG}" ]]; then
done echo ${LINK_CHECK_CONFIG} does not exist
exit 1
fi
# Switch for wikiscript branch="preview"
if [[ -d "doc.zih.tu-dresden.de" ]]; then if [[ -n "${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}" ]]; then
cd doc.zih.tu-dresden.de branch="origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}"
fi fi
mkdocs build function check_single_file(){
rm -rf public theFile="$1"
if [[ -e "${theFile}" ]]; then
echo "Checking links in ${theFile}"
if ! ${mlc} -q -c "${LINK_CHECK_CONFIG}" -p "${theFile}"; then
return 1
fi
else
echo "$1 not found"
fi
return 0
}
function check_files(){
any_fails=false
echo "Check files:"
echo "${files}"
echo ""
for f in ${files}; do
if ! check_single_file "${f}"; then
any_fails=true
fi
done
if [[ "${any_fails}" == true ]]; then
exit 1
fi
}
function check_changed_files(){
files=$(git diff --name-only "$(git merge-base HEAD "${branch}")" | grep '.md$' || true)
check_files
}
# Make use of the html-proofer-plugin and build documentation
function check_all_files_hpp() {
export ENABLED_HTMLPROOFER=true
export ENABLED_HTMLPROOFER_EXTERNAL_URLS=true
echo "Info: Building documentation and checking internal and external links. Might take some time."
# Switch for wikiscript
if [[ -d "doc.zih.tu-dresden.de" ]]; then
cd doc.zih.tu-dresden.de
fi
mkdocs build
rm -rf public
}
if [[ $# -ge 1 ]]; then
case $1 in
-h | help | -help | --help)
usage
exit
;;
-a | --all)
check_all_files_hpp
;;
-c | --changed)
check_changed_files
;;
*)
check_single_file "$1"
;;
esac
fi
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
Nutzungsbedingungen | Datenschutzerklärung | Barrierefreiheitserklärung