Commit 0fe61794 authored by Sasa Zhang's avatar Sasa Zhang
Browse files

newly modified README and config.json and test_suite.py

parent 6cc9524e
Pipeline #194508 passed with stage
in 52 seconds
{ {
"description": "This is the description message for case041.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case042.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case043.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case044.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case045.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case046.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case047.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case048.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
{ {
"description": "This is the description message for case049.", "description": "This is the description message.",
"addition_to_command_line": null, "addition_to_command_line": null,
"expected_running_result": "SUCCESS" "expected_running_result": "SUCCESS"
} }
\ No newline at end of file
-----------------------------------------------------------------------------------------
METRo test_suite README file METRo test_suite README file
-----------------------------------------------------------------------------------------
You are currently reading the test_suite README file of the release 4.0.0 (December 2019) You are currently reading the test_suite README file of the release 4.0.0 (December 2019)
...@@ -25,71 +27,70 @@ represents a case having case number 100). ...@@ -25,71 +27,70 @@ represents a case having case number 100).
============= What are all the possible files included inside a case directory? ============ ============= What are all the possible files included inside a case directory? ============
"config.json": config.json:
This is the configuration file and it includes three attributes: "description", This is the configuration file and it includes three attributes: "description",
"addition_to_command_line" and "expected_running_result". "addition_to_command_line" and "expected_running_result".
One extra attribute "error_torelance" can be added and the value of this attribute One extra attribute "error_tolerance" can be added and the value of this
has to be in decimal places (For example, having the value of "0.05" as the attribute has to be in decimal places (For example, having the value of "0.05"
"error_tolerance", the comparison result between "roadcast_test_suite_run.xml" and as the "error_tolerance", the comparison result between
"roadcast_reference.xml" lies outside the range of 0.05; in other words, any "roadcast_test_suite_run.xml" and "roadcast_reference.xml" lies outside the
comparison differences within 0.05 are ignored). range of 0.05; in other words, any comparison differences within 0.05 are ignored).
Note: If you would like to use the default error tolerance of "0.01", there is no Note: If you would like to use the default error tolerance of "0.01", there is no
need to add this extra attribute in "config.json". need to add this extra attribute in "config.json".
"forecast.xml", "observation.xml" and "station.xml": forecast.xml, observation.xml and station.xml:
These are the input files and each case should have these three files in order for These are the input files and each case should have these three files in order
METRo to generate possible forecast. for METRo to generate possible forecast.
"roadcast_reference.xml": roadcast_reference.xml:
This is a benchmark forecast file and it only exists for those cases whose attribute This is a benchmark forecast file and it only exists for those cases whose
"expected_running_result" has value of "SUCCESS". attribute "expected_running_result" has value of "SUCCESS".
=========================== How to run test_suite.py? ====================================== =========================== How to run test_suite.py? ======================================
1. Go to the directory "~/usr/share/metro/data/test_suite" where "~" represents the home 1. Go to the directory "~/usr/share/metro/data/test_suite" where "~" represents the home
directory of METRo. directory of METRo.
2. Options to enter the command: 2. Enter general command "python3 test_suite.py" to run all the cases inside the directory
2.1 The command "python3 test_suite.py": "test_suite" and displays the running results in a simplified version.
test_suite.py runs all cases inside the directory "test_suite" and displays
the running results in a simplified version.
2.2 The command "python3 test_suite.py -v": Possible parameters to run altogether with the general command:
test_suite.py runs all cases inside the directory "test_suite" and displays "-v":
the detailed information of running processes and outputs the running results test_suite.py runs all cases inside the directory "test_suite" and displays
in a verbose version. the detailed information of running processes and outputs the running results
in a verbose version.
Note: "-v" can be used in combination with either "-c" or "-s" and "-e". Note: "-v" can be used in combination with either "-c" or "-s" and "-e".
2.3 The command "python3 test_suite.py -c XX, XX, XX, ..." where XX represents the case number: "-c XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs the case(s) whose number(s) is/are entered after "-c". test_suite.py runs the case(s) whose number(s) is/are entered after "-c".
You can enter as many case numbers as you would like to run, but you have to You can enter as many case numbers as you would like to run, but you have to
enter at least one case number in order to active "-c". enter at least one case number in order to active "-c".
Note: "-c" and "-s" cannot be used simultaneously. Note: "-c" and "-s" cannot be used simultaneously.
2.4 The command "python3 test_suite.py -s XX, XX, XX, ..." where XX represents the case number: "-s XX, XX, XX, ..." where XX represents the case number:
test_suite.py runs by skipping the case(s) whose number(s) is/are entered after "-s". test_suite.py runs by skipping the case(s) whose number(s) is/are entered after
You can enter as many case numbers as you would like to skip, but you have to "-s". You can enter as many case numbers as you would like to skip, but you have
enter at least one case number in order to active "-s". to enter at least one case number in order to active "-s".
Note: "-c" and "-s" cannot be used simultaneously. Note: "-c" and "-s" cannot be used simultaneously.
2.5 The command "python3 test_suite.py -e YY" where YY represents the error tolerance in "-e YY" where YY represents the error tolerance in floating points:
floating points: test_suite.py runs all cases inside the directory "test_suite" and compares
test_suite.py runs all cases inside the directory "test_suite" and compares "roadcast_test_suite_run.xml" with "roadcast_reference.xml" and outputs
"roadcast_test_suite_run.xml" with "roadcast_reference.xml" and outputs comparison comparison result(s) using YY instead of the default value of 0.01 as error
result(s) using YY instead of the default value of 0.01 as error of tolerance. of tolerance.
Note: "-e" can be used in combination with either "-c" or "-s" and "-v". Note: "-e" can be used in combination with either "-c" or "-s" and "-v".
2.6 The command "python3 test_suite.py --clean": "--clean":
test_suite.py deletes all the files named "roadcast_test_suite_run.xml" within the test_suite.py deletes all the files named "roadcast_test_suite_run.xml" within
directory "test_suite". the directory "test_suite".
Note: "--clean" can be used in combination with "-v" which displays all the Note: "--clean" can be used in combination with "-v" which displays all the
directory/directories where the "roadcast_test_suite_run.xml" is/are deleted. directory/directories where the "roadcast_test_suite_run.xml" is/are deleted.
Note: You can always enter "python3 test_suite.py --help" for more information on how to use above Note: You can always enter "python3 test_suite.py --help" for more information on how to use
argument(s) to run test_suite.py above argument(s) to run test_suite.py
\ No newline at end of file \ No newline at end of file
...@@ -522,6 +522,9 @@ def main(): ...@@ -522,6 +522,9 @@ def main():
if expected_value == 'FAILURE': if expected_value == 'FAILURE':
print('>>>>>>>>>>>>>>>>>>>>>>>> {} is expected to FAIL...... <<<<<<<<<<<<<<<<<<' print('>>>>>>>>>>>>>>>>>>>>>>>> {} is expected to FAIL...... <<<<<<<<<<<<<<<<<<'
'<<<'.format(folder)) '<<<'.format(folder))
if expected_value == 'SUCCESS':
print('>>>>>>>>>>>>>>>>>>>>>>>> {} is expected to SUCCEED...... <<<<<<<<<<<<<<<<<<'
'<<<'.format(folder))
print('\nSyntax to run {}:'.format(folder)) print('\nSyntax to run {}:'.format(folder))
print('--------------------------------------------------------------------------------') print('--------------------------------------------------------------------------------')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment