FATE Flow Client¶

Description¶

Introduces how to install and use the FATE Flow Client, which is usually included in the FATE Client, which contains several clients of the FATE Project: Pipeline, FATE Flow Client and FATE Test.
Introducing the command line provided by FATE Flow Client, all commands will have a common invocation entry, you can type flow in the command line to get all the command categories and their subcommands.

    [IN]
    flow

    [OUT]
    Usage: flow COMMAND [OPTIONS]

      Fate Flow Client

    Options.
      -h, --help Show this message and exit.

    Commands: -h, --help
      Component Component Operations
      data Data Operations
      init Flow CLI Init Command
      Job Job Operations
      model Model Operations
      queue Queue Operations
      table Table Operations
      task Task Operations

For more information, please consult the following documentation or use the flow --help command.

All commands are described

Install FATE Client¶

Online installation¶

FATE Client will be distributed to pypi, you can install the corresponding version directly using tools such as pip, e.g.

pip install fate-client

or

pip install atmosphere-client==${version}

Installing on a FATE cluster¶

Please install on a machine with version 1.5.1 and above of FATE.

Installation command.

cd $FATE_PROJECT_BASE/
# Enter the virtual environment of FATE PYTHON
source bin/init_env.sh
# Execute the installation
cd fate/python/fate_client && python setup.py install

Once the installation is complete, type flow on the command line and enter, the installation will be considered successful if you get the following return.

Usage: flow [OPTIONS] COMMAND [ARGS]...

  Fate Flow Client

Options:
  -h, --help Show this message and exit.

Commands:
  component Component Operations
  data Data Operations
  init Flow CLI Init Command
  Job Job Operations
  model Model Operations
  queue Queue Operations
  Table Table Operations
  tag Tag Operations
  task Task Operations
Task Operations

## Initialization

Before using the fate-client, you need to initialize it. It is recommended to use the configuration file of fate-client to initialize it.

### Specify the fateflow service address

```bash
### Specify the IP address and port of the fateflow service for initialization
flow init --ip 192.168.0.1 --port 9380

via the configuration file on the FATE cluster¶

### Go to the FATE installation path, e.g. /data/projects/fate
cd $FATE_PROJECT_BASE/
flow init -c conf/service_conf.yaml

The initialization is considered successful if you get the following return.

{
    "retcode": 0,
    "retmsg": "Fate Flow CLI has been initialized successfully."
}

Verify¶

Mainly verify that the client can connect to the FATE Flow Server, e.g. try to query the current job status

flow job query

Usually the retcode in the return is 0.

{
    "data": [],
    "retcode": 0,
    "retmsg": "no job could be found"
}

If it returns something like the following, it means that the connection is not available, please check the network situation

{
    "retcode": 100,
    "retmsg": "Connection refused. Please check if the fate flow service is started"
}

Data¶

upload¶

Used to upload the input data for the modeling task to the storage system supported by fate

flow data upload -c ${conf_path}

Note: conf_path is the parameter path, the specific parameters are as follows

Options

parameter name	required	type	description
file	yes	string	data storage path
id_delimiter	yes	string	Data separator, e.g. ","
head	no	int	Whether the data has a table header
partition	yes	int	Number of data partitions
storage_engine	no	string	storage engine type, default "EGGROLL", also support "HDFS", "LOCALFS", "HIVE", etc.
namespace	yes	string	table namespace
table_name	yes	string	table name
storage_address	no	object	The storage address of the corresponding storage engine is required
use_local_data	no	int	The default is 1, which means use the data from the client's machine; 0 means use the data from the fate flow service's machine.
drop	no	int	Whether to overwrite uploads
extend_sid	no	bool	Whether to add a new column for uuid id, default False
auto_increasing_sid	no	bool	Whether the new id column is self-increasing (will only work if extend_sid is True), default False

mete information

parameter name	required	type	description
input_format	no	string	The format of the data (danse, svmlight, tag:value), used to determine
delimiter	no	string	The data separator, default ","
tag_with_value	no	bool	Valid for tag data format, whether to carry value
tag_value_delimiter	no	string	tag:value data separator, default ":"
with_match_id	no	bool	Whether or not to carry match id
with_match_id	no	object	The name of the id column, effective when extend_sid is enabled, e.g., ["email", "phone"]
id_range	no	object	For tag/svmlight format data, which columns are ids
exclusive_data_type	no	string	The format of the special type data columns
data_type	no	string	Column data type, default "float64
with_label	no	bool	Whether to have a label, default False
label_name	no	string	The name of the label, default "y"
label_type	no	string	Label type, default "int"

In version 1.9.0 and later, passing in the meta parameter will generate anonymous information about the feature. Example

eggroll

{
    "file": "examples/data/breast_hetero_guest.csv",
    "id_delimiter": ",",
    "head": 1,
    "partition": 10,
    "namespace": "experiment",
    "table_name": "breast_hetero_guest",
    "storage_engine": "EGGROLL"
}

hdfs

{
    "file": "examples/data/breast_hetero_guest.csv",
    "id_delimiter": ",",
    "head": 1,
    "partition": 10,
    "namespace": "experiment",
    "table_name": "breast_hetero_guest",
    "storage_engine": "HDFS"
}

localfs

{
    "file": "examples/data/breast_hetero_guest.csv",
    "id_delimiter": ",",
    "head": 1,
    "partition": 4,
    "namespace": "experiment",
    "table_name": "breast_hetero_guest",
    "storage_engine": "LOCALFS"
}

return parameters

parameter name	type	description
jobId	string	job id
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": {
        "board_url": "http://xxx.xxx.xxx.xxx:8080/index.html#/dashboard?job_id=202111081218319075660&role=local&party_id=0",
        "code": 0,
        "dsl_path": "/data/projects/fate/jobs/202111081218319075660/job_dsl.json",
        "job_id": "202111081218319075660",
        "logs_directory": "/data/projects/fate/logs/202111081218319075660",
        "message": "success",
        "model_info": {
            "model_id": "local-0#model",
            "model_version": "202111081218319075660"
        },
        "namespace": "experiment",
        "pipeline_dsl_path": "/data/projects/fate/jobs/202111081218319075660/pipeline_dsl.json",
        "runtime_conf_on_party_path": "/data/projects/fate/jobs/202111081218319075660/local/0/job_runtime_on_party_conf.json",
        "runtime_conf_path":"/data/projects/fate/jobs/202111081218319075660/job_runtime_conf.json",
        "table_name": "breast_hetero_host",
        "train_runtime_conf_path":"/data/projects/fate/jobs/202111081218319075660/train_runtime_conf.json"
    },
    "jobId": "202111081218319075660",
    "retcode": 0,
    "retmsg": "success"
}

upload-history¶

Used to query upload table history.

flow data upload-history -l 20
flow data upload-history --job-id $JOB_ID

Options

parameter name	required	type	description
-l --limit	no	int	Number of records to return. (default: 10)
-j --job_id	no	string	Job ID

download¶

Brief description:

Used to download data from within the fate storage engine to file format data

flow data download -c ${conf_path}

Note: conf_path is the parameter path, the specific parameters are as follows

Options

parameter name	required	type	description
output_path	yes	string	download_path
table_name	yes	string	fate table name
namespace	yes	int	fate table namespace

Example:

{
  "output_path": "/data/projects/fate/breast_hetero_guest.csv",
  "namespace": "experiment",
  "table_name": "breast_hetero_guest"
}

return parameters

parameter name	type	description
jobId	string	job id
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": {
        "board_url": "http://xxx.xxx.xxx.xxx:8080/index.html#/dashboard?job_id=202111081457135282090&role=local&party_id=0",
        "code": 0,
        "dsl_path": "/data/projects/fate/jobs/202111081457135282090/job_dsl.json",
        "job_id": "202111081457135282090",
        "logs_directory": "/data/projects/fate/logs/202111081457135282090",
        "message": "success",
        "model_info": {
            "model_id": "local-0#model",
            "model_version": "202111081457135282090"
        },
        "pipeline_dsl_path": "/data/projects/fate/jobs/202111081457135282090/pipeline_dsl.json",
        "runtime_conf_on_party_path": "/data/projects/fate/jobs/202111081457135282090/local/0/job_runtime_on_party_conf.json",
        "runtime_conf_path": "/data/projects/fate/jobs/202111081457135282090/job_runtime_conf.json",
        "train_runtime_conf_path": "/data/projects/fate/jobs/202111081457135282090/train_runtime_conf.json"
    },
    "jobId": "202111081457135282090",
    "retcode": 0,
    "retmsg": "success"
}

writer¶

Brief description:

Used to download data from the fate storage engine to the external engine or to save data as a new table

flow data writer -c ${conf_path}

Note: conf_path is the parameter path, the specific parameters are as follows

Options

parameter name	required	type	description
table_name	yes	string	fate table name
namespace	yes	int	fate table namespace
storage_engine	no	string	Storage type, e.g., MYSQL
address	no	object	storage_address
output_namespace	no	string	Save as a table namespace for fate
output_name	no	string	Save as fate's table name
**Note: storage_engine, address are combined parameters that provide storage to the specified engine.
output_namespace, output_name are also combined parameters, providing the function to save as a new table of the same engine**

Example:

{
  "table_name": "name1",
  "namespace": "namespace1",
  "output_name": "name2",
  "output_namespace": "namespace2"
}

return

parameter name	type	description
jobId	string	job id
retcode	int	return code
retmsg	string	return information
data	object	return data

Example

{
    "data": {
        "board_url": "http://xxx.xxx.xxx.xxx:8080/index.html#/dashboard?job_id=202201121235115028490&role=local&party_id=0",
        "code": 0,
        "dsl_path": "/data/projects/fate/fateflow/jobs/202201121235115028490/job_dsl.json",
        "job_id": "202201121235115028490",
        "logs_directory": "/data/projects/fate/fateflow/logs/202201121235115028490",
        "message": "success",
        "model_info": {
            "model_id": "local-0#model",
            "model_version": "202201121235115028490"
        },
        "pipeline_dsl_path": "/data/projects/fate/fateflow/jobs/202201121235115028490/pipeline_dsl.json",
        "runtime_conf_on_party_path": "/data/projects/fate/fateflow/jobs/202201121235115028490/local/0/job_runtime_on_party_conf.json",
        "runtime_conf_path":"/data/projects/fate/fateflow/jobs/202201121235115028490/job_runtime_conf.json",
        "train_runtime_conf_path": "/data/projects/fate/fateflow/jobs/202201121235115028490/train_runtime_conf.json"
    },
    "jobId": "202201121235115028490",
    "retcode": 0,
    "retmsg": "success"
}

Table¶

info¶

Query information about the fate table (real storage address, number, schema, etc.)

flow table info [options]

options

parameters	short-format	long-format	required	type	description
table_name	`-t`	`--table-name`	yes	string	fate table name
namespace	`-n`	`--namespace`	yes	string	fate table namespace

returns | parameter name | type | description | | :------ | :----- | -------- | | retcode | int | return code | | retmsg | string | return information | | data | object | return data |

Sample

{
    "data": {
        "address": {
            "home": null,
            "name": "breast_hetero_guest",
            "namespace": "experiment"
        },
        "count": 569,
        "exists": 1,
        "namespace": "experiment",
        "partition": 4,
        "schema": {
            "header": "y,x0,x1,x2,x3,x4,x5,x6,x7,x8,x9",
            "sid": "id"
        },
        "table_name": "breast_hetero_guest"
    },
    "retcode": 0,
    "retmsg": "success"
}

delete¶

You can delete table data with table delete

flow table delete [options]

Options

parameters	short-format	long-format	required	type	description
table_name	`-t`	`--table-name`	yes	string	fate table name
namespace	`-n`	`--namespace`	yes	string	fate table namespace

returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	object	return data

Sample

{
    "data": {
        "namespace": "xxx",
        "table_name": "xxx"
    },
    "retcode": 0,
    "retmsg": "success"
}

bind¶

Real storage addresses can be mapped to fate storage tables via table bind

flow table bind [options]

options

parameters	short format	long format	required	type	description
conf_path	`-c`	`--conf-path`	yes	string	configuration-path

Note: conf_path is the parameter path, the specific parameters are as follows

parameter_name	required	type	description
name	yes	string	fate table name
namespace	yes	string	fate table namespace
engine	yes	string	storage engine, supports "HDFS", "MYSQL", "PATH"
yes	object	real storage address
drop	no	int	Overwrite previous information
head	no	int	Whether there is a data table header
id_delimiter	no	string	Data separator
id_column	no	string	id field
feature_column	no	array	feature_field

mete information

parameter name	required	type	description
input_format	no	string	The format of the data (danse, svmlight, tag:value), used to determine
delimiter	no	string	The data separator, default ","
tag_with_value	no	bool	Valid for tag data format, whether to carry value
tag_value_delimiter	no	string	tag:value data separator, default ":"
with_match_id	no	bool	Whether or not to carry match id
with_match_id	no	object	The name of the id column, effective when extend_sid is enabled, e.g., ["email", "phone"]
id_range	no	object	For tag/svmlight format data, which columns are ids
exclusive_data_type	no	string	The format of the special type data columns
data_type	no	string	Column data type, default "float64
with_label	no	bool	Whether to have a label, default False
label_name	no	string	The name of the label, default "y"
label_type	no	string	Label type, default "int"

In version 1.9.0 and later, if the meta parameter is passed in during the table bind phase, no anonymous information about the feature is generated directly. The feature anonymization information of the original data will be updated after the data has passed through the reader component once

Sample

hdfs

{
    "namespace": "experiment",
    "name": "breast_hetero_guest",
    "engine": "HDFS",
    "address": {
        "name_node": "hdfs://fate-cluster",
        "path": "/data/breast_hetero_guest.csv"
    },
    "id_delimiter": ",",
    "head": 1,
    "partitions": 10
}

mysql

{
  "engine": "MYSQL",
  "address": {
    "user": "fate",
    "passwd": "fate",
    "host": "127.0.0.1",
    "port": 3306,
    "db": "experiment",
    "name": "breast_hetero_guest"
  },
  "namespace": "experiment",
  "name": "breast_hetero_guest",
  "head": 1,
  "id_delimiter": ",",
  "partitions": 10,
  "id_column": "id",
  "feature_column": "y,x0,x1,x2,x3,x4,x5,x6,x7,x8,x9"
}

PATH

{
    "namespace": "xxx",
    "name": "xxx",
    "engine": "PATH",
    "address": {
        "path": "xxx"
    }
}

return

parameter name	type	description
retcode	int	return code
retmsg	string	return information
data	object	return data

Sample

{
    "data": {
        "namespace": "xxx",
        "table_name": "xxx"
    },
    "retcode": 0,
    "retmsg": "success"
}

disable¶

Tables can be made unavailable by table disable

flow table disable [options]

Options

parameters	short-format	long-format	required	type	description
table_name	`-t`	`--table-name`	yes	string	fate table name
namespace	`-n`	`--namespace`	yes	string	fate table namespace

returns

parameter name	type	description
retcode	int	return code
retmsg	string	return information
data	object	return data

Sample

{
    "data": {
        "namespace": "xxx",
        "table_name": "xxx"
    },
    "retcode": 0,
    "retmsg": "success"
}

enable¶

Tables can be made available with table enable

flow table enable [options]

Options

parameters	short-format	long-format	required	type	description
table_name	`-t`	`--table-name`	yes	string	fate table name
namespace	`-n`	`--namespace`	yes	string	fate table namespace

returns

parameter name	type	description
retcode	int	return code
retmsg	string	return information
data	object	return data

Sample

{
    "data": [{
        "namespace": "xxx",
        "table_name": "xxx"
    }],
    "retcode": 0,
    "retmsg": "success"
}

disable-delete¶

Tables that are currently unavailable can be deleted with disable-delete

flow table disable-delete

return

parameter name	type	description
retcode	int	return-code
retmsg	string	return information
data	object	return data

Sample

{
  "data": [
    {
      "namespace": "xxx",
      "table_name": "xxx"
    },
    {
      "namespace": "xxx",
      "table_name": "xxx"
    }
  ],
  "retcode": 0,
  "retmsg": "success"
}

Job¶

submit¶

Build a federated learning job with two configuration files: job dsl and job conf, and submit it to the scheduler for execution

flow job submit [options]

Options

parameter name	required	type	description
-d, --dsl-path	yes	string	path to job dsl
-c, --conf-path	yes	string	job conf's path

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
jobId	string	Job ID
data	dict	return data
data.dsl_path	string	The path to the actual running dsl configuration generated by the system based on the submitted dsl content
data.runtime_conf_on_party_path	string	The system-generated path to the actual running conf configuration for each party based on the submitted conf content
data.dsl_path	string	The system-generated path to the actual running conf configuration for each party based on the submitted conf content
data.board_url	string	fateboard view address
data.model_info	dict	Model identification information

Example

{
    "data": {
        "board_url": "http://127.0.0.1:8080/index.html#/dashboard?job_id=202111061608424372620&role=guest&party_id=9999",
        "code": 0,
        "dsl_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/job_dsl.json",
        "job_id": "202111061608424372620",
        "logs_directory": "$FATE_PROJECT_BASE/logs/202111061608424372620",
        "message": "success",
        "model_info": {
            "model_id": "arbiter-10000#guest-9999#host-10000#model",
            "model_version": "202111061608424372620"
        },
        "pipeline_dsl_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/pipeline_dsl.json",
        "runtime_conf_on_party_path": "$FATE_FATE_PROJECT_BASE/jobs/202111061608424372620/guest/9999/job_runtime_on_party_conf.json",
        "runtime_conf_path":"$FATE_PROJECT_BASE/jobs/202111061608424372620/job_runtime_conf.json",
        "train_runtime_conf_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/train_runtime_conf.json"
    },
    "jobId": "202111061608424372620",
    "retcode": 0,
    "retmsg": "success"
}

rerun¶

Rerun a job

flow job rerun [options]

Options

| parameter name | required | type | description | | :------------- | :------- | :--- | ----------- |------- | | -j, --job-id | yes | string | job id path | | --cpn, --component-name | no | string | Specifies which component to rerun from, unspecified components will not be executed if they have no upstream dependencies on the specified component; if not specified, the entire job will be rerun | | --force | no | bool | The job will be rerun even if it succeeds; if not specified, the job will be skipped if it succeeds |

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
jobId	string	Job ID
data	dict	return data

Example

flow job rerun -j 202111031100369723120

flow job rerun -j 202111031100369723120 -cpn hetero_lr_0

flow job rerun -j 202111031100369723120 -cpn hetero_lr_0 --force

parameter-update¶

Update the job parameters

flow job parameter-update [options]

Options

parameter-name	required	type	description
-j, --job-id	yes	string	job id path
-c, --conf-path	yes	string	The contents of the job conf that needs to be updated, no need to fill in parameters that don't need to be updated

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
jobId	string	Job ID
data	dict	Returns the updated job conf

Example

Assuming that the job is updated with some of the execution parameters of the hetero_lr_0 component, the configuration file is as follows.

{
  "job_parameters": {
  },
  "component_parameters": {
    "common": {
      "hetero_lr_0": {
        "alpha": 0.02,
        "max_iter": 5
      }
    }
  }
}

Execution of the following command takes effect.

flow job parameter-update -j 202111061957421943730 -c examples/other/update_parameters.json

Execute the following command to rerun.

flow job rerun -j 202111061957421943730 -cpn hetero_lr_0 --force

stop¶

Cancels or terminates the specified job

Options

number	parameters	short format	long format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	yes	Job ID

Example

flow job stop -j $JOB_ID

query¶

Retrieve task information. Options

number	parameters	short-format	long-format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	no	Job ID
2	role	`-r`	`--role`	no	role
3	party_id	`-p`	`--party_id`	no	Party ID
4	status	`-s`	`--status`	No	Task status

Example

flow job query -r guest -p 9999 -s complete
flow job query -j $JOB_ID

view¶

Retrieve the job data view. Options

number	parameters	short-format	long-format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	yes	Job ID
2	role	`-r`	`--role`	no	role
3	party_id	`-p`	`--party_id`	no	Party ID
4	status	`-s`	`--status`	No	Task status

Example

flow job view -j $JOB_ID -s complete

config¶

Download the configuration file for the specified job to the specified directory.

Options

number	parameters	short-format	long-format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	yes	Job ID
2	role	`-r`	`--role`	yes	role
3	party_id	`-p`	`--party_id`	yes	Party ID
4	output_path	`-o`	`--output-path`	yes	output directory

Example

flow job config -j $JOB_ID -r host -p 10000 --output-path . /examples/

log¶

Download the log file of the specified job to the specified directory. Options

number	parameters	short-format	long-format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	yes	Job ID
2	output_path	`-o`	`--output-path`	yes	output directory

Example

flow job log -j JOB_ID --output-path . /examples/

list¶

Show the list of jobs. Options

number	parameters	short-format	long-format	required parameters	parameter description
1	limit	`-l`	`-limit`	no	Returns the number limit (default: 10)

Example

flow job list
flow job list -l 30

dsl¶

Predictive DSL file generator. Options

number	parameters	short-format	long-format	required parameters	parameter description
1	cpn_list		`-cpn-list`	No	List of user-specified component names
2	cpn_path		`-cpn-path`	No	User-specified path to a file with a list of component names
3	train_dsl_path		`-train-dsl-path`	yes	path to the training dsl file
4	output_path	`-o`	`--output-path`	no	output directory path

Example

flow job dsl --cpn-path fate_flow/examples/component_list.txt --train-dsl-path fate_flow/examples/test_hetero_lr_job_dsl.json

flow job dsl --cpn-path fate_flow/examples/component_list.txt --train-dsl-path fate_flow/examples/test_hetero_lr_job_dsl.json -o fate_flow /examples/

flow job dsl --cpn-list "dataio_0, hetero_feature_binning_0, hetero_feature_selection_0, evaluation_0" --train-dsl-path fate_flow/examples/ test_hetero_lr_job_dsl.json -o fate_flow/examples/

flow job dsl --cpn-list [dataio_0,hetero_feature_binning_0,hetero_feature_selection_0,evaluation_0] --train-dsl-path fate_flow/examples/ test_hetero_lr_job_dsl.json -o fate_flow/examples/

Task¶

query¶

Retrieve Task information

Options

number	parameters	short format	long format	required parameters	parameter description
1	job_id	`-j`	`--job_id`	no	Job ID
2	role	`-r`	`--role`	no	role
3	party_id	`-p`	`--party_id`	no	Party ID
4	component_name	`-cpn`	`--component_name`	no	component_name
5	status	`-s`	`--status`	No	Task status

Example

flow task query -j $JOB_ID -p 9999 -r guest
flow task query -cpn hetero_feature_binning_0 -s complete

list¶

Show the list of Tasks. Options

number	parameters	short format	long format	required parameters	parameter description
1	limit	`-l`	`-limit`	no	Returns a limit on the number of results (default: 10)

Example

flow task list
flow task list -l 25

Tracking¶

metrics¶

Get a list of all metrics names generated by a component task

flow tracking metrics [options]

Options

parameter name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter-name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data

Example

flow tracking metrics -j 202111081618357358520 -r guest -p 9999 -cpn evaluation_0

Output:

{
    "data": {
        "train": [
            "hetero_lr_0",
            "hetero_lr_0_ks_fpr",
            "hetero_lr_0_ks_tpr",
            "hetero_lr_0_lift",
            "hetero_lr_0_gain",
            "hetero_lr_0_accuracy",
            "hetero_lr_0_precision",
            "hetero_lr_0_recall",
            "hetero_lr_0_roc",
            "hetero_lr_0_confusion_mat",
            "hetero_lr_0_f1_score",
            "hetero_lr_0_quantile_pr"
        ]
    },
    "retcode": 0,
    "retmsg": "success"
}

metric-all¶

Get all the output metrics for a component task

flow tracking metric-all [options]

Options

parameter-name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter-name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking metric-all -j 202111081618357358520 -r guest -p 9999 -cpn evaluation_0

Output (limited space, only some of the metric data is shown and some values are omitted in the middle of the array type data):

{
    "data": {
        "train": {
            "hetero_lr_0": {
                "data": [
                    [
                        "auc",
                        0.293893
                    ],
                    [
                        "ks",
                        0.0
                    ]
                ],
                "meta": {
                    "metric_type": "EVALUATION_SUMMARY",
                    "name": "hetero_lr_0"
                }
            },
            "hetero_lr_0_accuracy": {
                "data": [
                    [
                        0.0,
                        0.372583
                    ],
                    [
                        0.99,
                        0.616872
                    ]
                ],
                "meta": {
                    "curve_name": "hetero_lr_0",
                    "metric_type": "ACCURACY_EVALUATION",
                    "name": "hetero_lr_0_accuracy",
                    "thresholds": [
                        0.999471,
                        0.002577
                    ]
                }
            },
            "hetero_lr_0_confusion_mat": {
                "data": [],
                "meta": {
                    "fn": [
                        357,
                        0
                    ],
                    "fp": [
                        0,
                        212
                    ],
                    "metric_type": "CONFUSION_MAT",
                    "name": "hetero_lr_0_confusion_mat",
                    "thresholds": [
                        0.999471,
                        0.0
                    ],
                    "tn": [
                        212,
                        0
                    ],
                    "tp": [
                        0,
                        357
                    ]
                }
            }
        }
    },
    "retcode": 0,
    "retmsg": "success"
}

parameters¶

After the job is submitted, the system resolves the actual component task parameters based on the component_parameters in the job conf combined with the system default component parameters

flow tracking parameters [options]

Options

parameter_name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter-name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking parameters -j 202111081618357358520 -r guest -p 9999 -cpn hetero_lr_0

Output:

{
    "data": {
        "ComponentParam": {
            "_feeded_deprecated_params": [],
            "_is_raw_conf": false,
            "_name": "HeteroLR#hetero_lr_0",
            "_user_feeded_params": [
                "batch_size",
                "penalty",
                "max_iter",
                "learning_rate",
                "init_param",
                "optimizer",
                "init_param.init_method",
                "alpha"
            ],
            "alpha": 0.01,
            "batch_size": 320,
            "callback_param": {
                "callbacks": [],
                "early_stopping_rounds": null,
                "metrics": [],
                "save_freq": 1,
                "use_first_metric_only": false,
                "validation_freqs": null
            },
            "cv_param": {
                "history_value_type": "score",
                "mode": "hetero",
                "n_splits": 5,
                "need_cv": false,
                "output_fold_history": true,
                "random_seed": 1,
                "role": "guest",
                "shuffle": true
            },
            "decay": 1,
            "decay_sqrt": true,
            "early_stop": "diff",
            "early_stopping_rounds": null,
            "encrypt_param": {
                "key_length": 1024,
                "method": "Paillier"
            },
            "encrypted_mode_calculator_param": {
                "mode": "strict",
                "re_encrypted_rate": 1
            },
            "floating_point_precision": 23,
            "init_param": {
                "fit_intercept": true,
                "init_const": 1,
                "init_method": "random_uniform",
                "random_seed": null
            },
            "learning_rate": 0.15,
            "max_iter": 3,
            "metrics": [
                "auc",
                "ks"
            ],
            "multi_class": "ovr",
            "optimizer": "rmsprop",
            "penalty": "L2",
            "predict_param": {
                "threshold": 0.5
            },
            "sqn_param": {
                "memory_M": 5,
                "random_seed": null,
                "sample_size": 5000,
                "update_interval_L": 3
            },
            "stepwise_param": {
                "direction": "both",
                "max_step": 10,
                "mode": "hetero",
                "need_stepwise": false,
                "nvmax": null,
                "nvmin": 2,
                "role": "guest",
                "score_name": "AIC"
            },
            "tol": 0.0001,
            "use_first_metric_only": false,
            "validation_freqs": null
        },
        "module": "HeteroLR"
    },
    "retcode": 0,
    "retmsg": "success"
}

output-data¶

Get the component output

flow tracking output-data [options]

options

parameter-name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl
-o, --output-path	yes	string	Path to output data

Returns

parameter name	type	description
retcode	int	Return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking output-data -j 202111081618357358520 -r guest -p 9999 -cpn hetero_lr_0 -o . /

Output :

{
    "retcode": 0,
    "directory": "$FATE_PROJECT_BASE/job_202111081618357358520_hetero_lr_0_guest_9999_output_data",
    "retmsg": "Download successfully, please check $FATE_PROJECT_BASE/job_202111081618357358520_hetero_lr_0_guest_9999_output_data directory "
}

output-data-table¶

Get the output data table name of the component

flow tracking output-data-table [options]

options

parameter-name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter-name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking output-data-table -j 202111081618357358520 -r guest -p 9999 -cpn hetero_lr_0

output:

{
    "data": [
        {
            "data_name": "train",
            "table_name": "9688fa00406c11ecbd0bacde48001122",
            "table_namespace": "output_data_202111081618357358520_hetero_lr_0_0"
        }
    ],
    "retcode": 0,
    "retmsg": "success"
}

output-model¶

Get the output model of a component task

flow tracking output-model [options]

options

parameter-name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter-name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking output-model -j 202111081618357358520 -r guest -p 9999 -cpn hetero_lr_0

Output:

{
    "data": {
        "bestIteration": -1,
        "encryptedWeight": {},
        "header": [
            "x0",
            "x1",
            "x2",
            "x3",
            "x4",
            "x5",
            "x6",
            "x7",
            "x8",
            "x9"
        ],
        "intercept": 0.24451607054764884,
        "isConverged": false,
        "iters": 3,
        "lossHistory": [],
        "needOneVsRest": false,
        "weight": {
            "x0": 0.04639947589856569,
            "x1": 0.19899685467216902,
            "x2": -0.18133550931649306,
            "x3": 0.44928868756862206,
            "x4": 0.05285905125502288,
            "x5": 0.319187932844076,
            "x6": 0.42578983446194013,
            "x7": -0.025765956309895477,
            "x8": -0.3699194462271593,
            "x9": -0.1212094750908295
        }
    },
    "meta": {
        "meta_data": {
            "alpha": 0.01,
            "batchSize": "320",
            "earlyStop": "diff",
            "fitIntercept": true,
            "learningRate": 0.15,
            "maxIter": "3",
            "needOneVsRest": false,
            "optimizer": "rmsprop",
            "partyWeight": 0.0,
            "penalty": "L2",
            "reEncryptBatches": "0",
            "revealStrategy": "",
            "tol": 0.0001
        },
        "module_name": "HeteroLR"
    },
    "retcode": 0,
    "retmsg": "success"
}

get-summary¶

Each component allows to set some summary information for easy observation and analysis

flow tracking get-summary [options]

Options

parameter-name	required	type	description
-j, --job-id	yes	string	job-id
-r, --role	yes	string	participant-role
-p, --partyid	yes	string	participant-id
-cpn, --component-name	yes	string	Component name, consistent with that in job dsl

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow tracking get-summary -j 202111081618357358520 -r guest -p 9999 -cpn hetero_lr_0

Output:

{
    "data": {
        "best_iteration": -1,
        "coef": {
            "x0": 0.04639947589856569,
            "x1": 0.19899685467216902,
            "x2": -0.18133550931649306,
            "x3": 0.44928868756862206,
            "x4": 0.05285905125502288,
            "x5": 0.319187932844076,
            "x6": 0.42578983446194013,
            "x7": -0.025765956309895477,
            "x8": -0.3699194462271593,
            "x9": -0.1212094750908295
        },
        "intercept": 0.24451607054764884,
        "is_converged": false,
        "one_vs_rest": false
    },
    "retcode": 0,
    "retmsg": "success"
}

tracking-source¶

For querying the parent and source tables of a table

flow table tracking-source [options]

Options

parameter-name	required	type	description
name	yes	string	fate table name
namespace	yes	string	fate table namespace

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": [{"parent_table_name": "61210fa23c8d11ec849a5254004fdc71", "parent_table_namespace": "output_data_202111031759294631020_hetero _lr_0_0", "source_table_name": "breast_hetero_guest", "source_table_namespace": "experiment"}],
    "retcode": 0,
    "retmsg": "success"
}

tracking-job¶

For querying the usage of a particular table

flow table tracking-job [options]

Options

parameter name	required	type	description
name	yes	string	fate table name
namespace	yes	string	fate table namespace

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": {"count":2, "jobs":["202111052115375327830", "202111031816501123160"]},
    "retcode": 0,
    "retmsg": "success"
}

Model¶

load¶

Load a model generated by deploy to Fate-Serving.

flow model load -c examples/model/publish_load_model.json
flow model load -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	Yes	Config file path
job_id	`-j`	`--job-id`	Yes	Job ID

Example

{
  "data": {
    "detail": {
      "guest": {
        "9999": {
          "retcode": 0,
          "retmsg": "success"
        }
      },
      "host": {
        "10000": {
          "retcode": 0,
          "retmsg": "success"
        }
      }
    },
    "guest": {
      "9999": 0
    },
    "host": {
      "10000": 0
    }
  },
  "jobId": "202111091122168817080",
  "retcode": 0,
  "retmsg": "success"
}

bind¶

Bind a model generated by deploy to Fate-Serving.

flow model bind -c examples/model/bind_model_service.json
flow model bind -c examples/model/bind_model_service.json -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
job_id	`-j`	`--job-id`	Yes	Job ID

Example

{
  "retcode": 0,
  "retmsg": "service id is 123"
}

import¶

Import the model from a file or storage engine.

flow model import -c examples/model/import_model.json
flow model import -c examples/model/restore_model.json --from-database

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
from_database		`--from-database`	Yes	Import the model from storage engine

Example

{
  "data": {
    "job_id": "202208261102212849780",
    "model_id": "arbiter-10000#guest-9999#host-10000#model",
    "model_version": "foobar",
    "party_id": "9999",
    "role": "guest"
  },
  "retcode": 0,
  "retmsg": "success"
}

export¶

Export the model to a file or storage engine.

flow model export -c examples/model/export_model.json
flow model export -c examples/model/store_model.json --to-database

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
to_database		`--to-database`	Yes	Export the model to storage engine

Example

{
  "data": {
    "board_url": "http://127.0.0.1:8080/index.html#/dashboard?job_id=202111091124582110490&role=local&party_id=0",
    "code": 0,
    "dsl_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/job_dsl.json",
    "job_id": "202111091124582110490",
    "logs_directory": "/root/Codes/FATE-Flow/logs/202111091124582110490",
    "message": "success",
    "model_info": {
      "model_id": "local-0#model",
      "model_version": "202111091124582110490"
    },
    "pipeline_dsl_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/pipeline_dsl.json",
    "runtime_conf_on_party_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/local/0/job_runtime_on_party_conf.json",
    "runtime_conf_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/job_runtime_conf.json",
    "train_runtime_conf_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/train_runtime_conf.json"
  },
  "jobId": "202111091124582110490",
  "retcode": 0,
  "retmsg": "success"
}

migrate¶

Migrate the model.

flow model migrate -c examples/model/migrate_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

Example

{
  "data": {
    "arbiter": {
      "10000": 0
    },
    "detail": {
      "arbiter": {
        "10000": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/arbiter#100#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      },
      "guest": {
        "9999": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/guest#99#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      },
      "host": {
        "10000": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/host#100#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      }
    },
    "guest": {
      "9999": 0
    },
    "host": {
      "10000": 0
    }
  },
  "jobId": "202111091127392613050",
  "retcode": 0,
  "retmsg": "success"
}

List tags of the model.

flow model tag-list -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
job_id	`-j`	`--job_id`	No	Job ID

tag-model¶

Add or remove a tag from the model.

flow model tag-model -j <job_id> -t <tag_name>
flow model tag-model -j <job_id> -t <tag_name> --remove

Options

Parameter	Short Flag	Long Flag	Optional	Description
job_id	`-j`	`--job_id`	No	Job ID
tag_name	`-t`	`--tag-name`	No	Tag name
remove		`--remove`	Yes	Remove the tag

deploy¶

Configure predict DSL.

flow model deploy --model-id <model_id> --model-version <model_version>

Options

Parameter	Long Flag	Optional	Description
model_id	`--model-id`	No	Model ID
model_version	`--model-version`	No	Model version
cpn_list	`--cpn-list`	Yes	Components list
cpn_path	`--cpn-path`	Yes	Load components list from a file
dsl_path	`--dsl-path`	Yes	Predict DSL file path
cpn_step_index	`--cpn-step-index`	Yes	Specify a checkpoint model to replace the pipeline model Use `:` to separate component name and step index E.g. `--cpn-step-index cpn_a:123`
cpn_step_name	`--cpn-step-name`	Yes	Specify a checkpoint model to replace the pipeline model. Use `:` to separate component name and step name E.g. `--cpn-step-name cpn_b:foobar`

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": {
    "model_id": "arbiter-9999#guest-10000#host-9999#model",
    "model_version": "202111032227378766180",
    "arbiter": {
      "party_id": 9999
    },
    "guest": {
      "party_id": 10000
    },
    "host": {
      "party_id": 9999
    },
    "detail": {
      "arbiter": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role arbiter 9999 success"
        }
      },
      "guest": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role guest 10000 success"
        }
      },
      "host": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role host 9999 success"
        }
      }
    }
  }
}

get-predict-dsl¶

Get predict DSL of the model.

flow model get-predict-dsl --model-id <model_id> --model-version <model_version> -o ./examples/

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
output_path	`-o`	`--output-path`	No	Output path

get-predict-conf¶

Get the template of predict config.

flow model get-predict-conf --model-id <model_id> --model-version <model_version> -o ./examples/

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
output_path	`-o`	`--output-path`	No	Output path

get-model-info¶

Get model information.

flow model get-model-info --model-id <model_id> --model-version <model_version>
flow model get-model-info --model-id <model_id> --model-version <model_version> --detail

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	Yes	Party role
party_id	`-p`	`--party-id`	Yes	Party ID
detail		`--detail`	Yes	Display detailed information

homo-convert¶

Convert trained homogenous model to the format of another ML framework.

flow model homo-convert -c examples/model/homo_convert_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

homo-deploy¶

Deploy trained homogenous model to a target online serving system. Currently the supported target serving system is KFServing.

flow model homo-deploy -c examples/model/homo_deploy_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

Checkpoint¶

list¶

List checkpoints.

flow checkpoint list --model-id <model_id> --model-version <model_version> --role <role> --party-id <party_id> --component-name <component_name>

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	No	Party role
party_id	`-p`	`--party-id`	No	Party ID
component_name	`-cpn`	`--component-name`	No	Component name

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": [
    {
      "create_time": "2021-11-07T02:34:54.683015",
      "step_index": 0,
      "step_name": "step_name",
      "models": {
        "HeteroLogisticRegressionMeta": {
          "buffer_name": "LRModelMeta",
          "sha1": "6871508f6e6228341b18031b3623f99a53a87147"
        },
        "HeteroLogisticRegressionParam": {
          "buffer_name": "LRModelParam",
          "sha1": "e3cb636fc93675684bff27117943f5bfa87f3029"
        }
      }
    }
  ]
}

get¶

Get checkpoint information.

flow checkpoint get --model-id <model_id> --model-version <model_version> --role <role> --party-id <party_id> --component-name <component_name> --step-index <step_index>

Example

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	No	Party role
party_id	`-p`	`--party-id`	No	Party ID
component_name	`-cpn`	`--component-name`	No	Component name
step_index		`--step-index`	Yes	Step index, cannot be used with `step_name`
step_name		`--step-name`	Yes	Step name, cannot be used with `step_index`

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": {
    "create_time": "2021-11-07T02:34:54.683015",
    "step_index": 0,
    "step_name": "step_name",
    "models": {
      "HeteroLogisticRegressionMeta": "CgJMMhEtQxzr4jYaPxkAAAAAAADwPyIHcm1zcHJvcDD///////////8BOTMzMzMzM8M/QApKBGRpZmZYAQ==",
      "HeteroLogisticRegressionParam": "Ig0KAng3EW1qASu+uuO/Ig0KAng0EcNi7a65ReG/Ig0KAng4EbJbl4gvVea/Ig0KAng2EcZwlVZTkOu/Ig0KAngwEVpG8dCbGvG/Ig0KAng5ESJNTx5MLve/Ig0KAngzEZ88H9P8qfO/Ig0KAng1EVfWP8JJv/K/Ig0KAngxEVS0xVXoTem/Ig0KAngyEaApgW32Q/K/KSiiE8AukPs/MgJ4MDICeDEyAngyMgJ4MzICeDQyAng1MgJ4NjICeDcyAng4MgJ4OUj///////////8B"
    }
  }
}

Provider¶

list¶

List all current component providers and information about the components they provide

flow provider list [options]

Options

Returns

参数名	类型	说明
retcode	int	返回码
retmsg	string	返回信息
data	dict	返回数据

Example

output:

{
    "data": {
        "fate": {
            "1.9.0": {
                "class_path": {
                    "anonymous_generator": "util.anonymous_generator_util.Anonymous",
                    "data_format": "util.data_format_preprocess.DataFormatPreProcess",
                    "feature_instance": "feature.instance.Instance",
                    "feature_vector": "feature.sparse_vector.SparseVector",
                    "hetero_model_merge": "protobuf.model_merge.merge_hetero_models.hetero_model_merge",
                    "homo_model_convert": "protobuf.homo_model_convert.homo_model_convert",
                    "interface": "components.components.Components",
                    "model": "protobuf.generated",
                    "model_migrate": "protobuf.model_migrate.model_migrate"
                },
                "components": [
                    "heterodatasplit",
                    "psi",
                    "heterofastsecureboost",
                    "heterofeaturebinning",
                    "scorecard",
                    "sampleweight",
                    "homosecureboost",
                    "onehotencoder",
                    "secureinformationretrieval",
                    "homoonehotencoder",
                    "datatransform",
                    "dataio",
                    "heterosshelinr",
                    "intersection",
                    "homofeaturebinning",
                    "secureaddexample",
                    "union",
                    "datastatistics",
                    "columnexpand",
                    "homonn",
                    "labeltransform",
                    "heterosecureboost",
                    "heterofeatureselection",
                    "heterolr",
                    "feldmanverifiablesum",
                    "heteropoisson",
                    "evaluation",
                    "federatedsample",
                    "homodatasplit",
                    "ftl",
                    "localbaseline",
                    "featurescale",
                    "featureimputation",
                    "heteropearson",
                    "heterokmeans",
                    "heteronn",
                    "heterolinr",
                    "spdztest",
                    "heterosshelr",
                    "homolr"
                ],
                "path": "${FATE_PROJECT_BASE}/python/federatedml",
                "python": ""
            },
            "default": {
                "version": "1.9.0"
            }
        },
        "fate_flow": {
            "1.9.0": {
                "class_path": {
                    "anonymous_generator": "util.anonymous_generator_util.Anonymous",
                    "data_format": "util.data_format_preprocess.DataFormatPreProcess",
                    "feature_instance": "feature.instance.Instance",
                    "feature_vector": "feature.sparse_vector.SparseVector",
                    "hetero_model_merge": "protobuf.model_merge.merge_hetero_models.hetero_model_merge",
                    "homo_model_convert": "protobuf.homo_model_convert.homo_model_convert",
                    "interface": "components.components.Components",
                    "model": "protobuf.generated",
                    "model_migrate": "protobuf.model_migrate.model_migrate"
                },
                "components": [
                    "writer",
                    "modelrestore",
                    "upload",
                    "apireader",
                    "modelstore",
                    "cacheloader",
                    "modelloader",
                    "download",
                    "reader"
                ],
                "path": "${FATE_FLOW_BASE}/python/fate_flow",
                "python": ""
            },
            "default": {
                "version": "1.9.0"
            }
        }
    },
    "retcode": 0,
    "retmsg": "success"
}

Contains the name, version number, codepath, list of provided components

register¶

Register a component provider

flow provider register [options]

Options

参数名	必选	类型	说明
-c, --conf-path	是	string	配置路径

Returns

参数名	类型	说明
retcode	int	返回码
retmsg	string	返回信息

Example

flow provider register -c $FATE_FLOW_BASE/examples/other/register_provider.json

conf:

{
  "name": "fate",
  "version": "1.7.1",
  "path": "${FATE_FLOW_BASE}/python/component_plugins/fateb/python/federatedml"
}

output:

{
    "data": {
        "flow-xxx-9380": {
            "retcode": 0,
            "retmsg": "success"
        }
    },
    "retcode": 0,
    "retmsg": "success"
}

Resource¶

query¶

For querying fate system resources

flow resource query

Options

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": {
        "computing_engine_resource": {
            "f_cores": 32,
            "f_create_date": "2021-09-21 19:32:59",
            "f_create_time": 1632223979564,
            "f_engine_config": {
                "cores_per_node": 32,
                "nodes": 1
            },
            "f_engine_entrance": "fate_on_eggroll",
            "f_engine_name": "EGGROLL",
            "f_engine_type": "computing",
            "f_memory": 0,
            "f_nodes": 1,
            "f_remaining_cores": 32,
            "f_remaining_memory": 0,
            "f_update_date": "2021-11-08 16:56:38",
            "f_update_time": 1636361798812
        },
        "use_resource_job": []
    },
    "retcode": 0,
    "retmsg": "success"
}

return¶

Resources for returning a job

flow resource return [options]

Options

parameter name	required	type	description
job_id	yes	string	job_id

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	object	return data

Example

{
    "data": [
        {
            "job_id": "202111081612427726750",
            "party_id": "8888",
            "resource_in_use": true,
            "resource_return_status": true,
            "role": "guest"
        }
    ],
    "retcode": 0,
    "retmsg": "success"
}

Privilege¶

grant¶

Add privileges

flow privilege grant -c fateflow/examples/permission/grant.json

Options

parameter name	required	type	description
party_id	yes	string	site id
component	no	string	component name, can be split by "," for multiple components, "*" for all components
dataset	no	object	list of datasets

sample

{
  "party_id": 10000,
  "component": "reader,dataio",
  "dataset": [
    {
      "namespace": "experiment",
      "name": "breast_hetero_guest"
    },
    {
      "namespace": "experiment",
      "name": "breast_hetero_host"
    }
  ]
}

return

parameter name	type	description
retcode	int	return code
retmsg	string	return message

Sample

{
    "retcode": 0,
    "retmsg": "success"
}

delete¶

Delete permissions

flow privilege delete -c fateflow/examples/permission/delete.json

Options

parameter name	required	type	description
party_id	yes	string	site_id
component	no	string	component name, can be split by "," for multiple components, "*" for all components
dataset	no	object	list of datasets, "*" is all datasets

sample

{
  "party_id": 10000,
  "component": "reader,dataio",
  "dataset": [
    {
      "namespace": "experiment",
      "name": "breast_hetero_guest"
    },
    {
      "namespace": "experiment",
      "name": "breast_hetero_host"
    }
  ]
}

return

parameter name	type	description
retcode	int	return code
retmsg	string	return message

Sample

{
    "retcode": 0,
    "retmsg": "success"
}

query¶

Query permissions

flow privilege query -p 10000

Options

parameters	short-format	long-format	required	type	description
party_id	`-p`	`--party-id`	yes	string	site id

returns

parameter name	type	description
retcode	int	return-code
retmsg	string	Return information
data	object	return data

Sample

{
    "data": {
        "component": [
            "reader",
            "dataio"
        ],
        "dataset": [
            {
                "name": "breast_hetero_guest",
                "namespace": "experiment"
            },
            {
                "name": "breast_hetero_host",
                "namespace": "experiment"
            }
        ]
    },
    "retcode": 0,
    "retmsg": "success"
}

Tag¶

create¶

Creates a label.

Options

number	parameters	short-format	long-format	required parameters	parameter description
1	tag_name	`-t`	`-tag-name`	yes	tag_name
2	tag_parameter_introduction	`-d`	`--tag-desc`	no	tag_introduction

Example

flow tag create -t tag1 -d "This is the parameter description of tag1."
flow tag create -t tag2

update¶

Update the tag information.

Options

number	parameters	short format	long format	required parameters	parameter description
1	tag_name	`-t`	`--tag-name`	yes	tag_name
2	new_tag_name		`--new-tag-name`	no	new-tag-name
3	new_tag_desc		`--new-tag-desc`	no	new tag introduction

Example

flow tag update -t tag1 --new-tag-name tag2
flow tag update -t tag1 --new-tag-desc "This is the introduction of the new parameter."

list¶

Show the list of tags.

options

number	parameters	short-format	long-format	required-parameters	parameter-introduction
1	limit	`-l`	`-limit`	no	Returns a limit on the number of results (default: 10)

Example

flow tag list
flow tag list -l 3

query¶

Retrieve tags.

Options

number	parameters	short-format	long-format	required parameters	parameter description
1	tag_name	`-t`	`-tag-name`	yes	tag_name
2	with_model		`-with-model`	no	If specified, information about models with this tag will be displayed

Example

flow tag query -t $TAG_NAME
flow tag query -t $TAG_NAME --with-model

delete¶

Delete the tag.

Options

number	parameters	short-format	long-format	required-parameters	parameters introduction
1	tag_name	`-t`	`---tag-name`	yes	tag_name

Example

flow tag delete -t tag1

Server¶

versions¶

List all relevant system version numbers

flow server versions

Options

None

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow server versions

Output:

{
    "data": {
        "API": "v1",
        "CENTOS": "7.2",
        "EGGROLL": "2.4.0",
        "FATE": "1.7.0",
        "FATEBoard": "1.7.0",
        "FATEFlow": "1.7.0",
        "JDK": "8",
        "MAVEN": "3.6.3",
        "PYTHON": "3.6.5",
        "SPARK": "2.4.1",
        "UBUNTU": "16.04"
    },
    "retcode": 0,
    "retmsg": "success"
}

reload¶

The following configuration items will take effect again after reload

All configurations after # engine services in $FATE_PROJECT_BASE/conf/service_conf.yaml
All configurations in $FATE_FLOW_BASE/python/fate_flow/job_default_config.yaml

flow server reload

Options

None

Returns

parameter name	type	description
retcode	int	return code
retmsg	string	return message
data	dict	return data
jobId	string	job id

Example

flow server reload

Output:

{
    "data": {
        "job_default_config": {
            "auto_retries": 0,
            "auto_retry_delay": 1,
            "default_component_provider_path": "component_plugins/fate/python/federatedml",
            "end_status_job_scheduling_time_limit": 300000,
            "end_status_job_scheduling_updates": 1,
            "federated_command_trys": 3,
            "federated_status_collect_type": "PUSH",
            "job_timeout": 259200,
            "max_cores_percent_per_job": 1,
            "output_data_summary_count_limit": 100,
            "remote_request_timeout": 30000,
            "task_cores": 4,
            "task_memory": 0,
            "task_parallelism": 1,
            "total_cores_overweight_percent": 1,
            "total_memory_overweight_percent": 1,
            "upload_max_bytes": 4194304000
        },
        "service_registry": null
    },
    "retcode": 0,
    "retmsg": "success"
}

FATE Flow Client¶

Description¶

Install FATE Client¶

Online installation¶

Installing on a FATE cluster¶

via the configuration file on the FATE cluster¶

Verify¶

Data¶

upload¶

upload-history¶

download¶

writer¶

Table¶

info¶

delete¶

bind¶

disable¶

enable¶

disable-delete¶

Job¶

submit¶

rerun¶

parameter-update¶

stop¶

query¶

view¶

config¶

log¶

list¶

dsl¶

Task¶

query¶

list¶

Tracking¶

metrics¶

metric-all¶

parameters¶

output-data¶

output-data-table¶

output-model¶

get-summary¶

tracking-source¶

tracking-job¶

Model¶

load¶

bind¶

import¶

export¶

migrate¶

tag-list¶

tag-model¶

deploy¶

get-predict-dsl¶

get-predict-conf¶

get-model-info¶

homo-convert¶

homo-deploy¶

Checkpoint¶

list¶

get¶

Provider¶

list¶

register¶

Resource¶

query¶

return¶

Privilege¶

grant¶

delete¶

query¶

Tag¶

create¶

update¶

list¶

query¶

delete¶

Server¶

versions¶

reload¶