Commit 0fe61794 authored by Sasa Zhang's avatar Sasa Zhang
Browse files

newly modified README and config.json and test_suite.py

parent 6cc9524e
Pipeline #194508 passed with stage
in 52 seconds
{
"description": "This is the description message for case041.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case042.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case043.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case044.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case045.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case046.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case047.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case048.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
{
"description": "This is the description message for case049.",
"description": "This is the description message.",
"addition_to_command_line": null,
"expected_running_result": "SUCCESS"
}
\ No newline at end of file
-----------------------------------------------------------------------------------------
METRo test_suite README file
-----------------------------------------------------------------------------------------
You are currently reading the test_suite README file of the release 4.0.0 (December 2019)
......@@ -25,71 +27,70 @@ represents a case having case number 100).
============= What are all the possible files included inside a case directory? ============
"config.json":
config.json:
This is the configuration file and it includes three attributes: "description",
"addition_to_command_line" and "expected_running_result".
One extra attribute "error_torelance" can be added and the value of this attribute
has to be in decimal places (For example, having the value of "0.05" as the
"error_tolerance", the comparison result between "roadcast_test_suite_run.xml" and
"roadcast_reference.xml" lies outside the range of 0.05; in other words, any
comparison differences within 0.05 are ignored).
One extra attribute "error_tolerance" can be added and the value of this
attribute has to be in decimal places (For example, having the value of "0.05"
as the "error_tolerance", the comparison result between
"roadcast_test_suite_run.xml" and "roadcast_reference.xml" lies outside the
range of 0.05; in other words, any comparison differences within 0.05 are ignored).
Note: If you would like to use the default error tolerance of "0.01", there is no
need to add this extra attribute in "config.json".
"forecast.xml", "observation.xml" and "station.xml":
These are the input files and each case should have these three files in order for
METRo to generate possible forecast.
forecast.xml, observation.xml and station.xml:
These are the input files and each case should have these three files in order
for METRo to generate possible forecast.
"roadcast_reference.xml":
This is a benchmark forecast file and it only exists for those cases whose attribute
"expected_running_result" has value of "SUCCESS".
roadcast_reference.xml:
This is a benchmark forecast file and it only exists for those cases whose
attribute "expected_running_result" has value of "SUCCESS".
=========================== How to run test_suite.py? ======================================
1. Go to the directory "~/usr/share/metro/data/test_suite" where "~" represents the home
directory of METRo.
2. Options to enter the command:
2.1 The command "python3 test_suite.py":
test_suite.py runs all cases inside the directory "test_suite" and displays
the running results in a simplified version.
2. Enter general command "python3 test_suite.py" to run all the cases inside the directory
"test_suite" and displays the running results in a simplified version.
2.2 The command "python3 test_suite.py -v":
test_suite.py runs all cases inside the directory "test_suite" and displays
the detailed information of running processes and outputs the running results
in a verbose version.
Possible parameters to run altogether with the general command:
"-v":
test_suite.py runs all cases inside the directory "test_suite" and displays
the detailed information of running processes and outputs the running results
in a verbose version.
Note: "-v" can be used in combination with either "-c" or "-s" and "-e".
Note: "-v" can be used in combination with either "-c" or "-s" and "-e".
2.3 The command "python3 test_suite.py -c XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs the case(s) whose number(s) is/are entered after "-c".
You can enter as many case numbers as you would like to run, but you have to
enter at least one case number in order to active "-c".
"-c XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs the case(s) whose number(s) is/are entered after "-c".
You can enter as many case numbers as you would like to run, but you have to
enter at least one case number in order to active "-c".
Note: "-c" and "-s" cannot be used simultaneously.
Note: "-c" and "-s" cannot be used simultaneously.
2.4 The command "python3 test_suite.py -s XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs by skipping the case(s) whose number(s) is/are entered after "-s".
You can enter as many case numbers as you would like to skip, but you have to
enter at least one case number in order to active "-s".
"-s XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs by skipping the case(s) whose number(s) is/are entered after
"-s". You can enter as many case numbers as you would like to skip, but you have
to enter at least one case number in order to active "-s".
Note: "-c" and "-s" cannot be used simultaneously.
Note: "-c" and "-s" cannot be used simultaneously.
2.5 The command "python3 test_suite.py -e YY" where YY represents the error tolerance in
floating points:
test_suite.py runs all cases inside the directory "test_suite" and compares
"roadcast_test_suite_run.xml" with "roadcast_reference.xml" and outputs comparison
result(s) using YY instead of the default value of 0.01 as error of tolerance.
"-e YY" where YY represents the error tolerance in floating points:
test_suite.py runs all cases inside the directory "test_suite" and compares
"roadcast_test_suite_run.xml" with "roadcast_reference.xml" and outputs
comparison result(s) using YY instead of the default value of 0.01 as error
of tolerance.
Note: "-e" can be used in combination with either "-c" or "-s" and "-v".
Note: "-e" can be used in combination with either "-c" or "-s" and "-v".
2.6 The command "python3 test_suite.py --clean":
test_suite.py deletes all the files named "roadcast_test_suite_run.xml" within the
directory "test_suite".
"--clean":
test_suite.py deletes all the files named "roadcast_test_suite_run.xml" within
the directory "test_suite".
Note: "--clean" can be used in combination with "-v" which displays all the
directory/directories where the "roadcast_test_suite_run.xml" is/are deleted.
Note: "--clean" can be used in combination with "-v" which displays all the
directory/directories where the "roadcast_test_suite_run.xml" is/are deleted.
Note: You can always enter "python3 test_suite.py --help" for more information on how to use above
argument(s) to run test_suite.py
\ No newline at end of file
Note: You can always enter "python3 test_suite.py --help" for more information on how to use
above argument(s) to run test_suite.py
\ No newline at end of file
......@@ -522,6 +522,9 @@ def main():
if expected_value == 'FAILURE':
print('>>>>>>>>>>>>>>>>>>>>>>>> {} is expected to FAIL...... <<<<<<<<<<<<<<<<<<'
'<<<'.format(folder))
if expected_value == 'SUCCESS':
print('>>>>>>>>>>>>>>>>>>>>>>>> {} is expected to SUCCEED...... <<<<<<<<<<<<<<<<<<'
'<<<'.format(folder))
print('\nSyntax to run {}:'.format(folder))
print('--------------------------------------------------------------------------------')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment