renku client --> renku workflows, activities, datasets
manage --> identify matching inputs, check for updates, "match" outputs
---
data:
name: "method_XY"
title: "example method XY"
description: "method to do ..."
keywords: ["method_omniexample"]
script: "src/run_method_XY.R"
benchmark_name: "omniexample"
inputs:
keywords: ["dataset_omniexample"]
files: ["counts", "meta_file"]
prefix:
counts: "_counts"
meta_file: "_meta"
outputs:
files:
method_res:
end: "json"
parameter:
names: ["dims", "n"]
keywords: ["parameter_omniexample"]
---
data:
...
script: "src/run_method_XY.R"
benchmark_name: "omniexample"
inputs:
keywords: ["dataset_omniexample"]
files: ["counts", "meta_file"]
prefix:
counts: "_counts"
meta_file: "_meta"
input_files:
explicit_dataset:
count_file: "data/explicit_dataset/explicit_dataset.mtx.gz"
meta_file: "data/explicit_dataset/explicit_dataset.json"
outputs:
...
---
data:
...
script: "src/run_method_XY.R"
benchmark_name: "omniexample"
inputs:
keywords: ["dataset_omniexample"]
files: ["counts", "meta_file"]
prefix:
counts: "_counts"
meta_file: "_meta"
filter_names: ["dataset1", "dataset2"]
outputs:
...
---
inputs:
...
outputs:
...
parameter:
names: ["dims", "n"]
keywords: ["parameter_omniexample"]
---
inputs:
...
outputs:
...
parameter:
values:
dims: [10, 20]
n: [10, 100, 1000]
---
inputs:
...
outputs:
...
parameter:
combinations:
comb1:
dims: 10
n: 10
comb2:
dims: 10
n: 100
comb3:
dims: 20
n: 10
---
inputs:
...
outputs:
...
parameter:
names: ["dims", "n"]
keywords: ["parameter_omniexample"]
filter:
dims:
upper: 100
lower: 10
n:
exclude: 100
Dataset X
true k: 3
Dataset Y
true k: 8
Parameter
k: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
Dataset X:
Dataset Y:
K
[1,2,3,4,5,6]
[5,6,7,8,9,10]
---
inputs:
...
outputs:
files:
method_res:
end: ".json"
---
inputs:
...
outputs:
file_mapping:
mapping1:
output_files:
method_res: "data/method_XY/method_XY_data1__dims_10__n_100.json"
input_files:
count_file: "data/data1/data1__counts.mtx.gz"
meta_file: "data/data1/data1__meta_file.json"
parameter:
dims: 10
n: 100
mapping2:
output_files:
method_res: "data/method_XY/method_XY_data2__dims_10__n_10.json"
input_files:
count_file: "data/data2/data2__counts.mtx.gz"
meta_file: "data/data2/data2__meta_file.json"
parameter:
dims: 10
n: 10
Dataset X
true k: 8
Dataset Y
true k: 3
Parameter
k: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
Dataset X:
Dataset Y:
K
[1,2,3,4,5,6]
[5,6,7,8,9,10]
Dataset ?:
[?]
Dataset ?
true k: ?
---
inputs:
...
outputs:
files:
method_res:
end: ".json"
filter_json: "path/to/filter_combinations.json"
import omnibenchmark as omni
omni_obj = omni.utils.get_omni_object_from_yaml("config.yaml")
omni_obj.__class__
<class 'omnibenchmark.core.omni_object.OmniObject'>
import omnibenchmark as omni
omni_obj = omni.utils.get_omni_object_from_yaml("config.yaml")
omni_obj.__dict__
{'name': 'method_XY',
'keyword': ['method_omniexample'],
'title': 'example method XY',
'description': 'method to do ...',
'command': <omnibenchmark.core.output_classes.OmniCommand object,
'inputs': <omnibenchmark.core.input_classes.OmniInput object,
'outputs': <omnibenchmark.core.output_classes.OmniOutput object,
'parameter': <omnibenchmark.core.input_classes.OmniParameter object,
'script': 'src/run_method_XY.R',
'omni_plan': None,
'orchestrator': 'https://renkulab.io/omniexample/orchestrator',
'benchmark_name': 'omniexample',
'wflow_name': 'method_XY',
'dataset_name': 'method_XY',
'renku': True,
'kg_url': 'https://renkulab.io/knowledge-graph'}
import omnibenchmark as omni
import filter_combinations
# Build object from config
omni_obj = omni.utils.get_omni_object_from_yaml("config.yaml")
omni_obj.__dict__
# Check for new inputs/updates
omni_obj.update_object()
# Generate json with filter combinations
filter_comb = filter_combinations.get_param_filter_by_ground_truth(omni_obj)
with open("src/filter_comb.json", "w") as fp:
json.dump(filter_comb, fp, indent=3)
renku_save()
# Update outputs and commands
omni_obj.outputs.filter_json = "src/filter_comb.json"
omni_obj.outputs.update_outputs()
omni_obj.command.outputs = omni_obj.outputs
omni_obj.command.update_command()
import omnibenchmark as omni
import filter_combinations
# Build object from config
omni_obj = omni.utils.get_omni_object_from_yaml("config.yaml")
omni_obj.__dict__
# Check for new inputs/updates
omni_obj.update_object()
# Generate json with filter combinations
filter_comb = filter_combinations.get_param_filter_by_ground_truth(omni_obj)
with open("src/filter_comb.json", "w") as fp:
json.dump(filter_comb, fp, indent=3)
renku_save()
## update outputs and commands
omni_obj.outputs.filter_json = "src/filter_comb.json"
omni_obj.outputs.update_outputs()
omni_obj.command.outputs = omni_obj.outputs
omni_obj.command.update_command()
3 main level filtering:
(renku-independent)
flexibility vs. simplicity?