#amqp #microservices #worker #message #sdk #media

mcai_worker_sdk

AMQP Worker to listen and provide trait to process message

60 releases (8 stable)

2.2.1 Aug 26, 2024
2.2.0 Jun 13, 2024
2.1.1-rc0 Oct 24, 2023
2.0.0-rc9 Jul 20, 2023
0.10.4 Jun 3, 2020

#465 in Web programming

Download history 33/week @ 2024-08-02 163/week @ 2024-08-23 28/week @ 2024-08-30 109/week @ 2024-09-06 65/week @ 2024-09-13 261/week @ 2024-09-20 57/week @ 2024-09-27 4/week @ 2024-10-04 13/week @ 2024-10-11 2/week @ 2024-10-18 54/week @ 2024-11-01 104/week @ 2024-11-08 28/week @ 2024-11-15

186 downloads per month
Used in 2 crates

MIT license

330KB
10K SLoC

MCAI Worker SDK

Build Status Coverage Status

MCAI Worker SDK contains library used for each worker defined in Media Cloud AI.
It defines a common way to exchange with Step manager developed in Elixir named StepFlow.

Documentation

Please use the auto-generated Rust documentation accessible here.


lib.rs:

MCAI Worker SDK

This library is an SDK to communicate via message broker with StepFlow. It's used for every worker as an abstraction. It manage itself requirements, message parsing, direct messaging.

Worker implementation

  1. Create a Rust project
  2. Add MCAI Worker SDK as a dependency in Cargo.toml: mcai_worker_sdk = "^1.0"
  3. Update the main file with the example provided here to implement MessageEvent trait, and call the start_worker to start the worker itself.
use mcai_worker_sdk::prelude::*;
use serde_derive::Deserialize;
use schemars::JsonSchema;

#[derive(Debug)]
struct WorkerNameEvent {}

#[derive(Debug, Deserialize, JsonSchema)]
struct WorkerParameters {}

// For opensource workers
default_rust_mcai_worker_description!();
// Or uncomment one of these lines for licensed workers
// default_rust_mcai_worker_description!(Private);
// default_rust_mcai_worker_description!(Commercial);
// You can also specify the name of your organisation
// default_rust_mcai_worker_description!("My Organisation", Commercial);

impl McaiWorker<WorkerParameters, RustMcaiWorkerDescription> for WorkerNameEvent {
}
static WORKER_NAME_EVENT: WorkerNameEvent = WorkerNameEvent {};

// uncomment it to start the worker
// fn main() {
//   mcai_worker_sdk::start_worker(&WORKER_NAME_EVENT);
// }

Runtime configuration

AMQP connection

Variable Description
AMQP_HOSTNAME IP or host of AMQP server (default: localhost)
AMQP_PORT AMQP server port (default: 5672)
AMQP_TLS enable secure connection using AMQPS (default: false, enable with true or 1 or TRUE or True)
AMQP_USERNAME Username used to connect to AMQP server (default: guest)
AMQP_PASSWORD Password used to connect to AMQP server (default: guest)
AMQP_VHOST AMQP virtual host (default: /)
AMQP_QUEUE AMQP queue name used to receive job orders (default: job_undefined)
AMQP_SERVER_CONFIGURATION RabbitMP configuration. Either standalone or cluster (default: standalone)

Vault connection

Variable Description
BACKEND_HOSTNAME URL used to connect to backend server (default: http://127.0.0.1:4000/api)
BACKEND_USERNAME Username used to connect to backend server
BACKEND_PASSWORD Password used to connect to backend server

Start worker locally

MCAI Worker SDK can be launched locally - without RabbitMQ. It can process some message for different purpose (functional tests, message order examples, etc.).

To start worker in this mode, setup the environment variable SOURCE_ORDERS with path(s) to json orders. It can take multiple orders, joined with : on unix platform, ; on windows os.

Examples:

RUST_LOG=info SOURCE_ORDERS=./examples/success_order.json:./examples/error_order.json cargo run --example worker

Dependencies

~31–47MB
~756K SLoC