Models¶

Instance model¶

class cornflow.models.InstanceModel(data)¶

Bases: BaseDataModel

Model class for the Instances It inherits from BaseDataModel to have the trace fields and user field

The InstanceModel has the following fields:

id: str, the primary key for the instances, a hash generated upon creation of the instance and the id given back to the user.The hash is generated from the creation time and the user id.
data: dict (JSON), the data structure of the instance (DataSchema)
name: str, the name given to the instance by the user.
description: str, the description given to the instance by the user. It is optional.
executions: relationship, not a field in the model but the relationship between the _class:InstanceModel and its dependent ExecutionModel.
user_id: int, the foreign key for the user (UserModel). It links the execution to its owner.
created_at: datetime, the datetime when the instance was created (in UTC). This datetime is generated automatically, the user does not need to provide it.
updated_at: datetime, the datetime when the instance was last updated (in UTC). This datetime is generated automatically, the user does not need to provide it.
deleted_at: datetime, the datetime when the instance was deleted (in UTC). Even though it is deleted, actually, it is not deleted from the database, in order to have a command that cleans up deleted data after a certain time of its deletion. This datetime is generated automatically, the user does not need to provide it.
data_hash: a hash of the data json using SHA256

Parameters:: data (dict) – the parsed json got from an endpoint that contains all the required information to create a new instance

update(data: dict)¶

Method used to update an instance from the database.

This method is mainly used on PUT requests to update the instance.

Parameters:: data (dict) – the data of the object
Returns:: Nothing, it will update the instance in-place and on the database
Return type:: None

Execution model¶

class cornflow.models.ExecutionModel(data)¶

Bases: BaseDataModel

Model class for the Executions. It inherits from BaseDataModel to have the trace fields and user field.

id: str, the primary key for the executions, a hash generated upon creation of the execution and the id given back to the user. The hash is generated from the creation date, the user and the id of the parent instance.
instance_id: str, the foreign key for the instance (InstanceModel). It links the execution to its parent instance.
name: str, the name of the execution given by the user.
description: str, the description of the execution given by the user. It is optional.
config: dict (JSON), the configuration to be used in the execution (ConfigSchema).
data: dict (JSON), the results from the execution (DataSchema).
log_text: text, the log generated by the airflow webserver during execution. This log is stored as text.
log_json: dict (JSON), the log generated by the airflow webserver during execution. This log is stored as a dict (JSON).
user_id: int, the foreign key for the user (UserModel). It links the execution to its owner.
created_at: datetime, the datetime when the execution was created (in UTC). This datetime is generated automatically, the user does not need to provide it.
updated_at: datetime, the datetime when the execution was last updated (in UTC). This datetime is generated automatically, the user does not need to provide it.
deleted_at: datetime, the datetime when the execution was deleted (in UTC). Even though it is deleted, actually, it is not deleted from the database, in order to have a command that cleans up deleted data after a certain time of its deletion. This datetime is generated automatically, the user does not need to provide it.
state: int, value representing state of the execution (finished, in progress, error, etc.)
state_message: str, a string value of state with human readable status message.
data_hash: a hash of the data json using SHA256

Parameters:: data (dict) – the parsed json got from an endpoint that contains all the required information to create a new execution

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

update(data)¶

Method used to update an execution from the database

Parameters:: data (dict) – the data of the object
Returns:: None
Return type:: None

update_config(config: dict)¶

Method to update the config of the execution after extending with default values

Parameters:: config (dict) – The config to store
Returns:: nothing

update_state(code, message=None)¶

Method to update the state code and message of an execution

Parameters:

code (int) – State code for the execution
message (str) – Message for the error

Returns:

nothing

update_log_txt(txt: str)¶

Method to update the log text of an execution

Parameters:: txt (str) – The log to store
Returns:: nothing

Case model¶

class cornflow.models.CaseModel(data, parent=None)¶

Bases: BaseDataModel

Model class for the Cases. It inherits from BaseDataModel to have the trace fields and user field.

id: int, the primary key for the cases, is an autoincrement.
path: str, the path for the case, by default it would be the “root folder” (empty string).
name: str, the name of the case given by the user.
description: str, the description of the case given by the user. It is optional.
data: dict (JSON), the data of the instance of the case.
checks: dict (JSON), the checks of instance of the case.
data_hash: str, the hash of the data of the instance of the case.
solution: dict (JSON), the solution of the instance of the case.
solution_hash: str, the hash of the solution of the instance of the case.
solution_checks: dict (JSON), the checks of the solution of the instance of the case.
schema: str, the schema of the instance of the case.
user_id: int, the foreign key for the user (UserModel). It links the case to its owner.
created_at: datetime, the datetime when the case was created (in UTC). This datetime is generated automatically, the user does not need to provide it.
updated_at: datetime, the datetime when the case was last updated (in UTC). This datetime is generated automatically, the user does not need to provide it.
deleted_at: datetime, the datetime when the case was deleted (in UTC). Even though it is deleted, actually, it is not deleted from the database, in order to have a command that cleans up deleted data after a certain time of its deletion.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

classmethod from_parent_id(user, data)¶

Class method to create a new case from an already existing case.

Parameters:

user (UserModel) – the user that is creating the case.
data (dict) – the data of the case

Returns:

the new case

Return type:

CaseModel

patch(data)¶

Method to patch the case

Parameters:: data (dict) – the patches to apply.

update(data)¶

Method used to update a case from the database

Parameters:: data (dict) – the data of the case
Returns:: None
Return type:: None

static apply_patch(original_data, data_patch)¶

Helper method to apply the patch and calculate the new hash

Parameters:

original_data (dict) – the dict with the original data
data_patch (list) – the list with the patch operations to perform

Returns:

the patched data and the hash

Return type:

Tuple(dict, str)

move_to(new_parent=None)¶

Method to move the case to a new path based on a new parent case

Parameters:: new_parent (CaseModel) – the new parent case

User model¶

class cornflow.models.UserModel(data)¶

Bases: TraceAttributesModel

Model class for the Users. It inherits from TraceAttributes to have trace fields.

The class UserModel has the following fields:

id: int, the user id, primary key for the users.
first_name: str, the name of the user.
last_name: str, the name of the user.
username: str, the username of the user used for the login.
email: str, the email of the user.
password: str, the hashed password of the user.
created_at: datetime, the datetime when the execution was created (in UTC). This datetime is generated automatically, the user does not need to provide it.
updated_at: datetime, the datetime when the execution was last updated (in UTC). This datetime is generated automatically, the user does not need to provide it.
deleted_at: datetime, the datetime when the execution was deleted (in UTC). Even though it is deleted, actually, it is not deleted from the database, in order to have a command that cleans up deleted data after a certain time of its deletion. This datetime is generated automatically, the user does not need to provide it.

Parameters:: data (dict) – the parsed json got from and endpoint that contains all the required information to create a new user.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

update(data)¶

Updates the user information in the database

Parameters:: data (dict) – the data to update the user

comes_from_external_provider()¶: Returns a boolean if the user comes from an external_provider or not

check_hash(password)¶

Method to check if the hash stored in the database is the same as the password given by the user

Parameters:: password (str) – the password given by the user.
Returns:: if the password is the same or not.
Return type:: bool

classmethod get_all_users()¶

Query to get all users

Returns:: a list with all the users.
Return type:: list(UserModel)

classmethod get_one_user(idx)¶

Query to get the information of one user

Parameters:: idx (int) – ID of the user
Returns:: the user object
Return type:: UserModel

classmethod get_one_user_by_email(email)¶

Query to get one user from the email

Parameters:: email (str) – User email
Returns:: the user object
Return type:: UserModel

classmethod get_one_user_by_username(username)¶

Returns one user (object) given a username

Parameters:: username (str) – the user username that we want to query for
Returns:: the user object
Return type:: UserModel

check_username_in_use()¶

Checks if a username is already in use

Returns:: a boolean if the username is in use
Return type:: bool

static generate_random_password() → str¶

Method to generate a new random password for the user

Returns:: the newly generated password
Return type:: str

is_admin()¶: Returns a boolean if a user is an admin or not

is_service_user()¶: Returns a boolean if a user is a service user or not