04990566a1 | ||
---|---|---|
.github/workflows | ||
api | ||
cli | ||
packages | ||
repo | ||
var/docs/images | ||
.gitignore | ||
docker-compose.dev.yaml | ||
docker-compose.yaml | ||
readme.md | ||
shell.nix |
readme.md
Piston
A high performance general purpose code execution engine.
About • Public API • Getting Started • Usage • Supported Languages • Principles • Security • License
About
Piston is a high performance general purpose code execution engine. It excels at running untrusted and possibly malicious code without fear from any harmful effects.
It's used in numerous places including:
- EMKC Challenges
- EMKC Weekly Contests
- Engineer Man Discord Server
- I Run Code (Discord Bot) bot as well as 4000+ other servers and 200+ direct integrations.
To get it in your own server, go here: https://emkc.org/run.
Official Extensions
The following are approved and endorsed extensions/utilities to the core Piston offering.
- Piston CLI, a universal shell supporting code highlighting, files, and interpretation without the need to download a language.
Public API
- Requires no installation and you can use it immediately.
- Reference the Versions/Execute sections below to learn about the request and response formats.
When using the public Piston API, use the base URL:
https://emkc.org/api/v1/piston
GET
https://emkc.org/api/v1/piston/versions
POST
https://emkc.org/api/v1/piston/execute
Important Note: The Piston API is rate limited to 5 requests per second. If you have a need for more requests than that and it's for a good cause, please reach out to me (EngineerMan#0001) on Discord so we can discuss potentially getting you an unlimited key.
Getting Started
All In One
Host System Package Dependencies
- Docker
- Docker Compose
- Node JS
After system dependencies are installed, clone this repository:
# clone and enter repo
git clone https://github.com/engineer-man/piston
Installation
# Start the API container
docker-compose up -d api
# Install all the dependencies for the cli
cd cli && npm i && cd -
Just Piston (no CLI)
Host System Package Dependencies
- Docker
Installation
docker run -v $PWD:'/piston' --tmpfs /piston/jobs -dit -p 2000:2000 --name piston_api ghcr.io/engineer-man/piston
Usage
CLI
The CLI is the main tool used for installing packages within piston, but also supports running code.
You can execute the cli with cli/index.js
.
# List all available packages
cli/index.js ppman list
# Install python 3.9.1
cli/index.js ppman install python 3.9.1
# Run a python script
echo 'print("Hello world!")' > test.py
cli/index.js run python 3.9.1 test.py
# Run the script using the latest version
cli/index.js run python '*' test.py
# Run using python 3.x
cli/index.js run python 3.x test.py
If you are operating on a remote machine, add the -u
flag like so:
cli/index.js -u http://piston.server:2000 ppman list
API
The container exposes an API on port 2000 by default. This is used by the CLI to carry out running jobs and package management.
Runtimes Endpoint
GET /api/v1/runtimes
This endpoint will return the supported languages along with the current version and aliases. To execute
code for a particular language using the /jobs
endpoint, either the name or one of the aliases must
be provided, along with the version.
Multiple versions of the same language may be present at the same time, and may be selected when running a job.
HTTP/1.1 200 OK
Content-Type: application/json
[
{
"language": "bash",
"version": "5.1.0",
"aliases": [
"sh"
]
},
{
"language": "brainfuck",
"version": "2.7.3",
"aliases": [
"bf"
]
},
...
]
Execute Endpoint
POST /api/v1/execute
This endpoint requests execution of some arbitrary code.
language
(required) The language to use for execution, must be a string and must be installed.version
(required) The version of the language to use for execution, must be a string containing a SemVer selector for the version or the specific version number to use.files
(required) An array of files containing code or other data that should be used for execution. The first file in this array is considered the main file.files[].name
(optinal) The name of the file to upload, must be a string containing no path.files[].content
(required) The content of the files to upload, must be a string containing text to write.stdin
(optional) The text to pass as stdin to the program. Must be a string, can be left blank.args
(optional) The arguments to pass to the program. Must be an array.compile_timeout
(optional) The maximum time allowed for the compile stage to finish before bailing out in milliseconds. Must be a number.run_timeout
(optional) The maximum time allowed for the run stage to finish before bailing out in milliseconds. Must be a number.
{
"language": "js",
"version": "15.10.0",
"files": [
{
"main": true,
"name": "my_cool_code.js",
"content": "console.log(process.argv)"
}
],
"stdin": "",
"args": [
"1",
"2",
"3"
],
"compile_timeout": 10000,
"run_timeout": 3000
}
A typical response upon successful execution will contain 1 or 2 keys run
and compile
.
compile
will only be present if the language requested requires a compile stage.
Each of these keys has an identical structure, containing both a stdout
and stderr
key, which is a string containing the text outputted during the stage into each buffer.
It also contains the code
and signal
which was returned from each process.
HTTP/1.1 200 OK
Content-Type: application/json
{
"run": {
"stdout": "[\n '/piston/packages/node/15.10.0/bin/node',\n '/piston/jobs/9501b09d-0105-496b-b61a-e5148cf66384/my_cool_code.js',\n '1',\n '2',\n '3'\n]\n",
"stderr": "",
"code": 0,
"signal": null
}
}
If a problem exists with the request, a 400
status code is returned and the reason in the message
key.
HTTP/1.1 400 Bad Request
Content-Type: application/json
{
"message": "html-5.0.0 runtime is unknown"
}
Supported Languages
bash
,
brainfuck
,
clojure
,
coffeescript
,
cow
,
crystal
,
dart
,
dash
,
deno
,
dotnet
,
elixir
,
emacs
,
erlang
,
gawk
,
gcc
,
go
,
groovy
,
haskell
,
java
,
jelly
,
julia
,
kotlin
,
lisp
,
lolcode
,
lua
,
mono
,
nasm
,
nim
,
node
,
ocaml
,
osabie
,
paradoc
,
pascal
,
perl
,
php
,
prolog
,
pure
,
python
,
rockstar
,
ruby
,
rust
,
scala
,
swift
,
typescript
,
vlang
,
zig
,
Principle of Operation
Piston uses Docker as the primary mechanism for sandboxing. There is an API within the container written in Node
which takes in execution requests and executees them within the container safely.
High level, the API writes any source code to a temporary directory in /piston/jobs
.
The source file is either ran or compiled and ran (in the case of languages like c, c++, c#, go, etc.).
Security
Docker provides a great deal of security out of the box in that it's separate from the system. Piston takes additional steps to make it resistant to various privilege escalation, denial-of-service, and resource saturation threats. These steps include:
- Disabling outgoing network interaction
- Capping max processes at 256 by default (resists
:(){ :|: &}:;
,while True: os.fork()
, etc.) - Capping max files at 2048 (resists various file based attacks)
- Cleaning up all temp space after each execution (resists out of drive space attacks)
- Running as a variety of unprivileged users
- Capping runtime execution at 3 seconds
- Capping stdout to 65536 characters (resists yes/no bombs and runaway output)
- SIGKILLing misbehaving code
License
Piston is licensed under the MIT license.