Docs

CROWDio — Distributed Compute Framework

!!! info “What is CROWDio?” CROWDio is an open-source distributed compute framework that lets developers run Python workloads across a crowd of heterogeneous worker nodes — including Android mobile devices — with zero infrastructure overhead.

Key Features

- :material-server-network: **Hub-and-Spoke Orchestration** A central **Foreman** service schedules and dispatches tasks to any number of connected workers over WebSocket. - :material-language-python: **Python-Native SDK** Submit functions with a single `await map(fn, iterable)` call. No refactoring required. - :material-android: **Mobile Worker Support** Android devices can join as workers via the Android app (Chaquopy-powered Python execution on-device). - :material-content-save-check: **Checkpointing & Recovery** Long-running tasks can emit state checkpoints and resume automatically on failure. - :material-tune: **Pluggable Schedulers** Built-in FIFO, Round-Robin, Least-Loaded, Priority, and advanced MCDM strategies (ARAS, EDAS, MABAC, WRR). - :material-chart-bar: **Built-in Evaluation** Benchmarking suites for scalability, heterogeneity, energy constraints, and failure simulation.

System at a Glance

+------------------------+       WebSocket (:9000)       +------------------------+
|   SDK Client App       | <---------------------------> |      Foreman           |
| (developer_sdk/api.py) |                               | (FastAPI + Scheduler)  |
+------------------------+                               +------------------------+
                                                                |
                                               Assign tasks / collect results
                                                                |
                                              +---------------------------------+
                                              |  Workers (Desktop + Android)    |
                                              |  pc_worker/  |  Android App      |
                                              +---------------------------------+
                                                                |
                                              +---------------------------------+
                                              |  SQLite (crowdcompute.db)       |
                                              +---------------------------------+

Documentation Map

Section What you’ll find
Getting Started Prerequisites, install, env setup
Quick Start Run Foreman + Worker + Client in 5 minutes
Architecture Component diagrams, data flow, message protocol
SDK Guide map, run, submit, get, pipeline with examples
Checkpointing Decorator API, recovery flow, REST endpoints
Operations Scheduler config, monitoring, DB operations
API Reference All REST and WebSocket API endpoints
Mobile Worker Android worker setup, build, and runtime
Evaluation Benchmarks and experiment suites

Quick Navigation

=== “I want to run a job NOW” → Quick Start

=== “I’m integrating the SDK” → SDK Guide

=== “I’m setting up a worker machine” → Quick Start — Worker Setup

=== “I want to use an Android phone as a worker” → Android Worker Guide

=== “I need to understand the architecture” → Architecture Overview