Federated Model Registry¶

1. Description¶

Models trained by FATE are automatically saved locally and recorded in the FATE-Flow database. models saved after each component run are called Pipeline models, and models saved at regular intervals while the component is running are called Checkpoint models. checkpoint models can also be used for retrying after a component run is unexpectedly interrupted The Checkpoint model can also be used for "breakpoints" when a component is retrying after an unexpected interruption.

Checkpoint model support has been added since 1.7.0 and is not saved by default. To enable it, add the callback ModelCheckpoint to the DSL.

Local disk storage¶

Pipeline models are stored in model_local_cache/<party_model_id>/<model_version>/variables/data/<component_name>/<model_alias>.
Checkpoint models are stored in model_local_cache/<party_model_id>/<model_version>/checkpoint/<component_name>/<step_index>#<step_name>.

Remote storage engine¶

Local disk is not reliable, so there is a risk of losing models. FATE-Flow supports exporting models to specified storage engines, importing from specified storage engines, and pushing models to engine storage when publishing models automatically.

The storage engine supports Tencent Cloud Object Storage, MySQL and Redis, please refer to Storage Engine Configuration

2. Model¶

load¶

Load a model generated by deploy to Fate-Serving.

flow model load -c examples/model/publish_load_model.json
flow model load -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	Yes	Config file path
job_id	`-j`	`--job-id`	Yes	Job ID

Example

{
  "data": {
    "detail": {
      "guest": {
        "9999": {
          "retcode": 0,
          "retmsg": "success"
        }
      },
      "host": {
        "10000": {
          "retcode": 0,
          "retmsg": "success"
        }
      }
    },
    "guest": {
      "9999": 0
    },
    "host": {
      "10000": 0
    }
  },
  "jobId": "202111091122168817080",
  "retcode": 0,
  "retmsg": "success"
}

bind¶

Bind a model generated by deploy to Fate-Serving.

flow model bind -c examples/model/bind_model_service.json
flow model bind -c examples/model/bind_model_service.json -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
job_id	`-j`	`--job-id`	Yes	Job ID

Example

{
  "retcode": 0,
  "retmsg": "service id is 123"
}

import¶

Import the model from a file or storage engine.

flow model import -c examples/model/import_model.json
flow model import -c examples/model/restore_model.json --from-database

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
from_database		`--from-database`	Yes	Import the model from storage engine

Example

{
  "data": {
    "job_id": "202208261102212849780",
    "model_id": "arbiter-10000#guest-9999#host-10000#model",
    "model_version": "foobar",
    "party_id": "9999",
    "role": "guest"
  },
  "retcode": 0,
  "retmsg": "success"
}

export¶

Export the model to a file or storage engine.

flow model export -c examples/model/export_model.json
flow model export -c examples/model/store_model.json --to-database

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path
to_database		`--to-database`	Yes	Export the model to storage engine

Example

{
  "data": {
    "board_url": "http://127.0.0.1:8080/index.html#/dashboard?job_id=202111091124582110490&role=local&party_id=0",
    "code": 0,
    "dsl_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/job_dsl.json",
    "job_id": "202111091124582110490",
    "logs_directory": "/root/Codes/FATE-Flow/logs/202111091124582110490",
    "message": "success",
    "model_info": {
      "model_id": "local-0#model",
      "model_version": "202111091124582110490"
    },
    "pipeline_dsl_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/pipeline_dsl.json",
    "runtime_conf_on_party_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/local/0/job_runtime_on_party_conf.json",
    "runtime_conf_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/job_runtime_conf.json",
    "train_runtime_conf_path": "/root/Codes/FATE-Flow/jobs/202111091124582110490/train_runtime_conf.json"
  },
  "jobId": "202111091124582110490",
  "retcode": 0,
  "retmsg": "success"
}

migrate¶

Migrate the model.

flow model migrate -c examples/model/migrate_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

Example

{
  "data": {
    "arbiter": {
      "10000": 0
    },
    "detail": {
      "arbiter": {
        "10000": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/arbiter#100#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      },
      "guest": {
        "9999": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/guest#99#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      },
      "host": {
        "10000": {
          "retcode": 0,
          "retmsg": "Migrating model successfully. The Config of model has been modified automatically. New model id is: arbiter-100#guest-99#host-100#model, model version is: 202111091127392613050. Model files can be found at '/root/Codes/FATE-Flow/temp/fate_flow/host#100#arbiter-100#guest-99#host-100#model_202111091127392613050.zip'."
        }
      }
    },
    "guest": {
      "9999": 0
    },
    "host": {
      "10000": 0
    }
  },
  "jobId": "202111091127392613050",
  "retcode": 0,
  "retmsg": "success"
}

List tags of the model.

flow model tag-list -j <job_id>

Options

Parameter	Short Flag	Long Flag	Optional	Description
job_id	`-j`	`--job_id`	No	Job ID

tag-model¶

Add or remove a tag from the model.

flow model tag-model -j <job_id> -t <tag_name>
flow model tag-model -j <job_id> -t <tag_name> --remove

Options

Parameter	Short Flag	Long Flag	Optional	Description
job_id	`-j`	`--job_id`	No	Job ID
tag_name	`-t`	`--tag-name`	No	Tag name
remove		`--remove`	Yes	Remove the tag

deploy¶

Configure predict DSL.

flow model deploy --model-id <model_id> --model-version <model_version>

Options

Parameter	Long Flag	Optional	Description
model_id	`--model-id`	No	Model ID
model_version	`--model-version`	No	Model version
cpn_list	`--cpn-list`	Yes	Components list
cpn_path	`--cpn-path`	Yes	Load components list from a file
dsl_path	`--dsl-path`	Yes	Predict DSL file path
cpn_step_index	`--cpn-step-index`	Yes	Specify a checkpoint model to replace the pipeline model Use `:` to separate component name and step index E.g. `--cpn-step-index cpn_a:123`
cpn_step_name	`--cpn-step-name`	Yes	Specify a checkpoint model to replace the pipeline model. Use `:` to separate component name and step name E.g. `--cpn-step-name cpn_b:foobar`

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": {
    "model_id": "arbiter-9999#guest-10000#host-9999#model",
    "model_version": "202111032227378766180",
    "arbiter": {
      "party_id": 9999
    },
    "guest": {
      "party_id": 10000
    },
    "host": {
      "party_id": 9999
    },
    "detail": {
      "arbiter": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role arbiter 9999 success"
        }
      },
      "guest": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role guest 10000 success"
        }
      },
      "host": {
        "party_id": {
          "retcode": 0,
          "retmsg": "deploy model of role host 9999 success"
        }
      }
    }
  }
}

get-predict-dsl¶

Get predict DSL of the model.

flow model get-predict-dsl --model-id <model_id> --model-version <model_version> -o ./examples/

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
output_path	`-o`	`--output-path`	No	Output path

get-predict-conf¶

Get the template of predict config.

flow model get-predict-conf --model-id <model_id> --model-version <model_version> -o ./examples/

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
output_path	`-o`	`--output-path`	No	Output path

get-model-info¶

Get model information.

flow model get-model-info --model-id <model_id> --model-version <model_version>
flow model get-model-info --model-id <model_id> --model-version <model_version> --detail

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	Yes	Party role
party_id	`-p`	`--party-id`	Yes	Party ID
detail		`--detail`	Yes	Display detailed information

homo-convert¶

Convert trained homogenous model to the format of another ML framework.

flow model homo-convert -c examples/model/homo_convert_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

homo-deploy¶

Deploy trained homogenous model to a target online serving system. Currently the supported target serving system is KFServing.

flow model homo-deploy -c examples/model/homo_deploy_model.json

Options

Parameter	Short Flag	Long Flag	Optional	Description
conf_path	`-c`	`--conf-path`	No	Config file path

3. Checkpoint¶

list¶

List checkpoints.

flow checkpoint list --model-id <model_id> --model-version <model_version> --role <role> --party-id <party_id> --component-name <component_name>

Options

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	No	Party role
party_id	`-p`	`--party-id`	No	Party ID
component_name	`-cpn`	`--component-name`	No	Component name

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": [
    {
      "create_time": "2021-11-07T02:34:54.683015",
      "step_index": 0,
      "step_name": "step_name",
      "models": {
        "HeteroLogisticRegressionMeta": {
          "buffer_name": "LRModelMeta",
          "sha1": "6871508f6e6228341b18031b3623f99a53a87147"
        },
        "HeteroLogisticRegressionParam": {
          "buffer_name": "LRModelParam",
          "sha1": "e3cb636fc93675684bff27117943f5bfa87f3029"
        }
      }
    }
  ]
}

get¶

Get checkpoint information.

flow checkpoint get --model-id <model_id> --model-version <model_version> --role <role> --party-id <party_id> --component-name <component_name> --step-index <step_index>

Example

Parameter	Short Flag	Long Flag	Optional	Description
model_id		`--model-id`	No	Model ID
model_version		`--model-version`	No	Model version
role	`-r`	`--role`	No	Party role
party_id	`-p`	`--party-id`	No	Party ID
component_name	`-cpn`	`--component-name`	No	Component name
step_index		`--step-index`	Yes	Step index, cannot be used with `step_name`
step_name		`--step-name`	Yes	Step name, cannot be used with `step_index`

Example

{
  "retcode": 0,
  "retmsg": "success",
  "data": {
    "create_time": "2021-11-07T02:34:54.683015",
    "step_index": 0,
    "step_name": "step_name",
    "models": {
      "HeteroLogisticRegressionMeta": "CgJMMhEtQxzr4jYaPxkAAAAAAADwPyIHcm1zcHJvcDD///////////8BOTMzMzMzM8M/QApKBGRpZmZYAQ==",
      "HeteroLogisticRegressionParam": "Ig0KAng3EW1qASu+uuO/Ig0KAng0EcNi7a65ReG/Ig0KAng4EbJbl4gvVea/Ig0KAng2EcZwlVZTkOu/Ig0KAngwEVpG8dCbGvG/Ig0KAng5ESJNTx5MLve/Ig0KAngzEZ88H9P8qfO/Ig0KAng1EVfWP8JJv/K/Ig0KAngxEVS0xVXoTem/Ig0KAngyEaApgW32Q/K/KSiiE8AukPs/MgJ4MDICeDEyAngyMgJ4MzICeDQyAng1MgJ4NjICeDcyAng4MgJ4OUj///////////8B"
    }
  }
}

4. Storage engine configuration¶

`enable_model_store`¶

This option affects API /model/load.

Automatic upload models to the model store if it exists locally but does not exist in the model storage, or download models from the model store if it does not exist locally but does not exist in the model storage.

This option does not affect API /model/store or /model/restore.

`model_store_address`¶

This config defines which storage engine to use.

Tencent Cloud Object Storage¶

storage: tencent_cos
# get these configs from Tencent Cloud console
Region:
SecretId:
SecretKey:
Bucket:

MySQL¶

storage: mysql
database: fate_model
user: fate
password: fate
host: 127.0.0.1
port: 3306
# other optional configs send to the engine
max_connections: 10
stale_timeout: 10

Redis¶

storage: redis
host: 127.0.0.1
port: 6379
db: 0
password:
# the expiry time of keys, in seconds. defaults None (no expiry time)
ex:

Federated Model Registry¶

1. Description¶

Local disk storage¶

Remote storage engine¶

2. Model¶

load¶

bind¶

import¶

export¶

migrate¶

tag-list¶

tag-model¶

deploy¶

get-predict-dsl¶

get-predict-conf¶

get-model-info¶

homo-convert¶

homo-deploy¶

3. Checkpoint¶

list¶

get¶

4. Storage engine configuration¶

enable_model_store¶

model_store_address¶

Tencent Cloud Object Storage¶

MySQL¶

Redis¶

`enable_model_store`¶

`model_store_address`¶