Commit b10e8e78 authored by Julius Metz's avatar Julius Metz

small quality changes

parent c8e45f02
......@@ -366,10 +366,11 @@ def parse_file(path, collectl, shorten_cmds, coarsest, config):
"""
collectl_starttime = time.time()
# run collectl in playback mode and read output into list
print('start collectl replay')
process = subprocess.run(
[collectl, '-P', '-p', path, '-sZ'], stdout=subprocess.PIPE, check=True,
)
print('collectl make table took {:.1f}s'.format(
print('collectl replay took {:.1f}s'.format(
time.time() - collectl_starttime))
parsing_starttime = time.time()
# output contains all data!
......@@ -716,18 +717,19 @@ def data_from_file(arguments):
@click.option('--plotlypath', '-p', type=click.Path(exists=True), help='path to plotly.js')
@click.option('--destination', '-d', default='.', type=click.Path(exists=True), show_default=True, help='path to directory where directory with plots will be created')
@click.option('--configpath', default=None, type=click.Path(exists=True), help='python file with plot and merge infos see doku for detail')
@click.option('--shorten/--notshorten', default=True, help='commands will be shorted only to name')
@click.option('--shorten/--noshorten', default=True, help='commands will be shorted only to name')
@click.option('--coarsest', is_flag=True, help='commands will be shorted only to type (bash, perl, ...)')
@click.option('--filtercmds/--notfiltercmds', default=True, help='filtering or not')
@click.option('--filtercmds/--nofiltercmds', default=True, help='filtering or not')
@click.option('--filtervalue', help='Parameter which is given to the filter.')
@click.option('--filtertype',
type=click.Choice(FILTER_FUNCTIONS.keys(), case_sensitive=False),
default=DEFAULT_FILTER, show_default=True,
help='Filter which is to be used.',
)
@click.option('--force', is_flag=True, help='override existing plot directory if exist')
@click.option('--force', '-f', is_flag=True, help='override existing plot directory if exist')
@click.option('--showcommands', is_flag=True, help='print all commands that are in the Plot')
def main(source, collectl, plotlypath, destination, configpath,
shorten, coarsest, filtercmds, filtervalue, filtertype, force):
shorten, coarsest, filtercmds, filtervalue, filtertype, force, showcommands):
source_paths = get_sources(source)
if not source_paths:
......@@ -829,5 +831,11 @@ def main(source, collectl, plotlypath, destination, configpath,
print("write/build of websites took {:.1f}s".format(time.time() - time_sites))
if showcommands:
print('commands:')
for command in cmd_colors.keys():
print(command)
if __name__ == '__main__':
main()
# Collectl2plotly
TODO: what is collectl2plotly? add link to http://collectl.sourceforge.net
Collectl2plotly is a python commandline tool, to build htmlsites with plotly diagrams from data that was collected with [collectl](http://collectl.sourceforge.net).
Collectl wiki is [here](http://collectl.sourceforge.net).
## How to
To use you need to install it via pip:
To use you need to install it via pip and collectl must be installed:
```
pip install ./Collectl2plotly
```
......@@ -32,21 +35,28 @@ You can also use collectl data from multiple cluster nodes that was recorded at
| -p, --plotlypath | a path to a plotly javascript libary | is needed for the plot in the html files | plotly.js next to skript |
| -d, --destination | a path to a directory | is where a directory with the html files will be created | current directory |
| --configpath | a path to config file | is a python file with the plot and merge settings see [here](#config)| a default config |
| --shorten /<br> --notshorten | - | enable or disable shorten of commands with parameters/options only to file/command names. <br> examples:<br> python ~/scripts/script.py 1 --> script.py <br>ls -lisa --> ls | enabled |
| --shorten /<br> --noshorten | - | enable or disable shorten of commands with parameters/options only to file/command names. <br> examples:<br> python ~/scripts/script.py 1 --> script.py <br>ls -lisa --> ls | enabled |
| --coarsest | - | enable shorten of commands only to command names.<br> If enabled --shorten is ignored!<br> examples: <br> python ~/scripts/script.py 1 --> python <br> ls -lisa --> ls | disabled |
| --filtercmds / --notfiltercmds | - | enable or disable filtering | enabled |
| --filtercmds / --nofiltercmds | - | enable or disable [filtering](#filter) | enabled |
| --filtertype | filtertype <br> see --help for detail | to select which filter to be used | see --help |
| --filtervalue | any value (string, int, ...) | is passed to the filter function as string | - |
| --force | - | override existing plot directory if exist | - |
| --showcommands | - | print all commands of the plots to copy for static colors | - |
| -f, --force | - | override existing plot directory if exist | - |
## Config
The config is a python file with the following varibles to create special plots or/and with different values instead of the default.
Usually, the default should be ok and there is no need to write a custom config.
TODO: describe what merging is
In the config you define how the values are merged.
Merging means that all values from "collectl" will be accumulated (using a merging function) to already existing values of the command/time point or a base value, if no values exist for that command/time point. By accumulating the values, they are also converted and formatted.
Note:
Filtering is not done in the config you filter in the [options](#options) of the command.
TODO: describe what filtering is
Filtering means sort out commands that are not relevant enough to be displayed in the diagrams to keep the diagramm clean.
#### NEEDED_VALUES
Specifies which values are merged under which names in a dict.
......@@ -205,6 +215,7 @@ PLOTLY_STATIC_COLORS = {}
```
## Filter
Filters are used to sort out commands that are not relevant enough to be displayed in the diagrams.
### Existing filter
| Name | Name of function | Description | filtervalue default |
......@@ -274,9 +285,9 @@ A filter function must return a dict, structured like this:
```
## Merger
A merger is a function that merge values from collectl to existing base value. It is also used to format values from collectl.
A merger is a function that merge values from collectl to existing values or the base value this is set in the config see [here](NEEDED_VALUES).
TODO: not clear what a base value is.
It is also used to format and convert the values from collectl.
### Existing merger
......@@ -289,7 +300,7 @@ TODO: not clear what a base value is.
| possible argument | take | describtion | supported merger |
| ----------------- | ---- | ----------- | ---------------- |
| round | int | Rounding merged collectl values on given point (Python round function). | x2oneaddition_* |
| operator | basic python operater as str (/ , * , -, +, %) | Specifies which operator should be used to calculate a value from parameter operator_value and the collectl values who already merged. This will be merged to the base value. | x2oneaddition_* |
| operator | basic python operater as str (/ , * , -, +, %) | Specifies which operator should be used to calculate a value from parameter operator_value and the collectl values. This will be merged to the base value. | x2oneaddition_* |
| operator_value | int/float | value that must be given to use operator for use see operator. | x2oneaddition_* |
### Add new Merger
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment