From e9a14e8a47a784b63eac979210d8d99783c71b76 Mon Sep 17 00:00:00 2001 From: Josh Uy Date: Sat, 16 Jan 2021 10:01:35 -0700 Subject: [PATCH 1/4] README rewrite --- readme.md | 241 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 172 insertions(+), 69 deletions(-) diff --git a/readme.md b/readme.md index 5b2cb85..ca541ed 100644 --- a/readme.md +++ b/readme.md @@ -1,68 +1,175 @@ -## Piston +

Piston

+

+
+ engineer-man piston +

+ +

A high performance general purpose code execution engine.

+
+ +

+ + GitHub last commit + + GitHub issues + + GitHub pull requests +

+ +--- + +

+ About • + Public API • + Getting Started • + Usage • + Supported Languages • + Principles • + Security • + License +

+ +--- +
+ +# About + +

Piston is a high performance general purpose code execution engine. It excels at running untrusted and possibly malicious code without fear from any harmful effects. -It's used in numerous places including -[EMKC Challenges](https://emkc.org/challenges), -[EMKC Weekly Contests](https://emkc.org/contests), the -[Engineer Man Discord Server](https://discord.gg/engineerman) via -[I Run Code](https://github.com/engineer-man/piston-bot) bot as well as 1300+ other servers -and 100+ direct integrations. To get it in your own server, go here: https://emkc.org/run. +

+
-#### Use Public API -Requires no installation and you can use it immediately. Reference the Versions/Execute sections -below to learn about the request and response formats. -- `GET` `https://emkc.org/api/v1/piston/versions` -- `POST` `https://emkc.org/api/v1/piston/execute` +It's used in numerous places including: +* [EMKC Challenges](https://emkc.org/challenges), +* [EMKC Weekly Contests](https://emkc.org/contests), +* [Engineer Man Discord Server](https://discord.gg/engineerman), +* [I Run Code (Discord Bot)](https://github.com/engineer-man/piston-bot) bot as well as 1300+ other servers +and 100+ direct integrations. -Important Note: The Piston API is rate limited to 5 requests per second. If you have a need for more requests than that +To get it in your own server, go here: https://emkc.org/run. + +
+ +# Public API + +- Requires no installation and you can use it immediately. +- Reference the Versions/Execute sections below to learn about the request and response formats. + +
+ +When using the public Piston API, use the base URL: + +``` +https://emkc.org/api/v1/piston +``` + +#### GET +``` +https://emkc.org/api/v1/piston/versions +``` +#### POST +``` +https://emkc.org/api/v1/piston/execute +``` + +> Important Note: The Piston API is rate limited to 5 requests per second. If you have a need for more requests than that and it's for a good cause, please reach out to me (EngineerMan#0001) on [Discord](https://discord.gg/engineerman) so we can discuss potentially getting you an unlimited key. -#### Cloning and System Dependencies -``` -# clone and enter repo -git clone https://github.com/engineer-man/piston -cd piston/lxc +
+ +# Getting Started + +### Host System Package Dependencies + +* NodeJS +* lxc +* libvirt + +
+ +If your OS is not documented below, please open pull requests with the correct commands for your OS. + +
+CentOS / RHEL + +```sh +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash +nvm install --lts +nvm use --lts -# centos/rhel dependencies: yum install -y epel-release yum install -y lxc lxc-templates debootstrap libvirt systemctl start libvirtd +``` +
-# ubuntu server 18.04 dependencies: -apt install lxc lxc-templates debootstrap libvirt0 +
+Ubuntu (18.04) + +```sh +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash +nvm install --lts +nvm use --lts + +apt install -y lxc lxc-templates debootstrap libvirt0 +``` +
+ +
+Arch Linux + +```sh +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash +nvm install --lts +nvm use --lts -# arch dependencies: sudo pacman -S lxc libvirt unzip +``` +
-# everything else: -# not documented, please open pull requests with commands for debian/arch/macos/etc +#### After system dependencies are installed, clone this repository: + +```sh +# clone and enter repo +git clone https://github.com/engineer-man/piston +cd piston/lxc ``` #### Installation (simple) -Coming soon. -#### Installation (advanced/manual) -See `var/install.txt` for how to create a new LXC container and install all of the required +- Coming soon. + +#### Installation (advanced) + +- See `var/install.txt` for how to create a new LXC container and install all of the required software. -#### CLI Usage -- `lxc/execute [language] [file path] [args]` +
-#### API Usage +# Usage + +### CLI + +```sh +lxc/execute [language] [file path] [args] +``` + +### API To use the API, it must first be started. Please note that if root is required to access LXC then the API must also be running as root. To start the API, run the following: + ``` cd api ./start ``` -#### Base URLs -When using the public Piston API, use: -``` -https://emkc.org/api/v1/piston -``` -For your own local installation, use: +For your own local installation, the API is available at + ``` http://127.0.0.1:2000 ``` @@ -135,44 +242,38 @@ Content-Type: application/json } ``` -#### Supported Languages -- awk -- bash -- brainfuck -- c -- cpp -- csharp -- deno -- erlang -- elixir -- emacs -- elisp -- go -- haskell -- java -- jelly -- julia -- kotlin -- lua -- nasm -- node -- paradoc -- perl -- php -- python2 -- python3 -- ruby -- rust -- swift -- typescript +
-#### Principle of Operation +# Supported Languages +| | | +|:---------:|------------| +| awk | julia | +| bash | kotlin | +| brainfuck | lua | +| c | nasm | +| cpp | node | +| csharp | paradoc | +| deno | perl | +| erlang | php | +| elixir | python2 | +| emacs | python3 | +| elisp | ruby | +| go | rust | +| haskell | swift | +| java | typescript | +| jelly | | + +
+ +# Principle of Operation Piston utilizes LXC as the primary mechanism for sandboxing. There is a small API written in Node which takes in execution requests and executes them in the container. High level, the API writes a temporary source and args file to `/tmp` and that gets mounted read-only along with the execution scripts into the container. The source file is either ran or compiled and ran (in the case of languages like c, c++, c#, go, etc.). -#### Security +
+ +# Security LXC provides a great deal of security out of the box in that it's separate from the system. Piston takes additional steps to make it resistant to various privilege escalation, denial-of-service, and resource saturation threats. These steps include: @@ -186,5 +287,7 @@ various privilege escalation, denial-of-service, and resource saturation threats - Capping stdout to 65536 characters (resists yes/no bombs and runaway output) - SIGKILLing misbehaving code -#### License +
+ +# License Piston is licensed under the MIT license. From 4c04ebdee0a531acdefdd9460b9fba81d954b29e Mon Sep 17 00:00:00 2001 From: Brian Seymour Date: Sun, 17 Jan 2021 14:45:58 -0600 Subject: [PATCH 2/4] Update readme.md --- readme.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/readme.md b/readme.md index ca541ed..625537d 100644 --- a/readme.md +++ b/readme.md @@ -1,7 +1,9 @@ -

Piston

-
- engineer-man piston + engineer-man piston + Piston +

+

+

A high performance general purpose code execution engine.

From f041385f34b0d0777129c1fc0bd57e103d4b257b Mon Sep 17 00:00:00 2001 From: Brian Seymour Date: Sun, 17 Jan 2021 14:46:17 -0600 Subject: [PATCH 3/4] Update readme.md --- readme.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/readme.md b/readme.md index 625537d..0702805 100644 --- a/readme.md +++ b/readme.md @@ -2,9 +2,6 @@ engineer-man piston Piston -

- -

A high performance general purpose code execution engine.


From 8de614ee9435e9792872ad317e9684f31c3d63f3 Mon Sep 17 00:00:00 2001 From: Brian Seymour Date: Thu, 21 Jan 2021 13:47:25 -0600 Subject: [PATCH 4/4] escape backslash --- lxc/executors/java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lxc/executors/java b/lxc/executors/java index 6fc09e0..1ad5cad 100755 --- a/lxc/executors/java +++ b/lxc/executors/java @@ -1,6 +1,6 @@ cd /tmp/$2 cp code.code interim.java -name=$(grep -Po "(?<=\n|\A)\s*(public\s+)?(class|interface)\s+\K([^\/\\\n\s{]+)" interim.java) +name=$(grep -Po "(?<=\n|\A)\s*(public\s+)?(class|interface)\s+\K([^\/\\\\\n\s{]+)" interim.java) mv interim.java $name.java runuser runner$1 -c "\