~cyberia/etc

capsul web application blog post v1 PROPOSED

forest
forest: 1
 capsul web application blog post

 2 files changed, 226 insertions(+), 0 deletions(-)
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git:

curl -s https://lists.cyberia.club/~cyberia/etc/patches/29/mbox | git am -3
Learn more about email & git
View this thread in the archives

[PATCH] capsul web application blog post Export this patch

forest
From: forestjohnson <forest.n.johnson@gmail.com>

---
 ...sul-rollin-onward-with-web-application.txt | 218 ++++++++++++++++++
 site/blog/blog.xml                            |   8 +
 2 files changed, 226 insertions(+)
 create mode 100644 site/blog/20200505-capsul-rollin-onward-with-web-application.txt

diff --git a/site/blog/20200505-capsul-rollin-onward-with-web-application.txt b/site/blog/20200505-capsul-rollin-onward-with-web-application.txt
new file mode 100644
index 0000000..b4ff68a
--- /dev/null
+++ b/site/blog/20200505-capsul-rollin-onward-with-web-application.txt
@@ -0,0 +1,218 @@

                         CAPSUL

        rollin' onwards with a web application


Forest                                         2020-05-20

                WHAT'S NEW IN CAPSUL?

Capsul has been operated by hand so far, with business 
conducted via email. Obviously, this isn't the best 
user experience. If no one is on the other end at the 
time, the user might feel as if they are shouting into 
the void.

Ideally, users could pay for service, create and destroy
capsuls, and monitor their capsul's status at any time. 

So we set out to create an application enabling that,
while keeping things as simple as possible. As of today,
you can experience it firsthand!

     -->   https://beta.capsul.cyberia.club/   <--


      WHAT IS CAPSUL? WHY WOULD ANYONE DO THAT?

Capsul started out as a "for fun" project to host 
multiple VMs with different operating systems on the same
physical server. 

A cloud compute provider experiment to find out:

 - How hard is it to build basic compute-as-service
   functionality that has been mythologized and 
   commoditized by some of the biggest software 
   businesses of all time.

 - What problems have to be solved in order to do
   this at a small scale?

 - And last but not least, 
   how much better-than-the-big-boys can we do? :P


I heard about Capsul and I thought, cool, why not.  

At first, I was slightly dismissive of the project --
why re-invent the wheel? There are lots of established 
tools for creating cloud services already out there, 
surely they would be hard for us to measure up to.

Of course, you could argue, that's not the point.
It's all about the journey, popping the hood and learning
how things are put together. 

But on the other hand, Capsul is something that we want 
to use, not just a pet project.


               Can I depend on it? 


            /⁀\                 ___/‾⁀|__,
         (‾‾____‾‾)        |      xx . .|
            /  \           |       [   >)    
           /    \          |       ‶` ‾
                           |               
      I WANT TO BELIEVE    |    (X)  DOUBT

Whether excited or doubtful, the tone of the question
expresses the real utility and risk associated with DIY.  

We have to **make our own seat belts** for this, an
experience and practice that I personally feel is highly
under-rated. 

I don't want to give up and just leave it to the experts.

I want to build the confidence necessary to make my own
systems, and to measure thier stability and efficacy. 

                        (\_/)
                        [. .]
                       ==<.>==
   
                 " Anyone can Cook "

It probably helps that I've never seen a friend get hurt 
because of a flaw in something I designed, but even if
I had, I'd like to think that I'd continue believing
in the idea that technology is never "beyond" us. 
I could never make it through Technoholics Anonymous,
because I'd never be able believe a Higher Power will
restore sanity to the machine and save us from ourselves.


           ABOUT THE DEVELOPMENT PROCESS

First step was to chose a language and framework. 
We made this decision (Python 3, Flask) almost entirely 
based on which language was the most commonly known in 
our group. I was the only one who had never used Python 
before, and I felt up to the task of learning a language 
as a part of this process. 

Next, we had to decide how the system would work. 

How would we secure user's accounts?  How would users 
pay for capsuls?  Would it be like a subscription, 
would you buy compute credits, or a recieve a bill at
the end of the month?

In the interest of simplicity, we opted to use a 
tumblr-style magic-link login instead of requiring 
the user to provide a password. So, you have to
recieve an email and click a link in that email
every time you log in. 

We also decided to go with the "purchase credits, then 
create capsul" payment workflow, because it was the 
easiest way we could accept both credit card and 
cryptocurrency payments, and we believed that requiring 
the user to pay first was an appropriate level of 
friction for our service, at least right now.

I had never worked on a project that integrated 
with a payment processor or had a "dollars" column in a 
database table before. I felt like I worked at the
Federal Reserve, typing 

INSERT INTO payments (account, dollars) VALUES
    ('forest', 20.00);

into my database during development. 

The application has three backends:

 - a postgres database where all of the payment and 
   account data is stored
   
 - the virtualization backend which lifecycles the 
   virtual machines and provides information about them
   (whether or not they exist, and current IP address)

 - Prometheus metrics database which allows the
   web application to display real-time metrics for each
   capsul.

All of the payments are handled by external payment 
processors Stripe and BTCPay Server, so the application
doesn't have to deal with credit cards or cryptocurrency
directly. What's even better, because BTCPay Server 
tracks the status of invoices automatically, we can
accept unconfirmed transactions as valid payments and
then rewind the payment if we learn that it was a
double-spend attack. No need to bother the user about 
Replace By Fee or anything like that. 

The initial development phase took one week. Some days
I worked on it for 12+ hours, I think. I was having a 
blast. I believe that the application should be secure
against common types of attacks. I kept the OWASP
Top 10 Web Application Security Risks in mind while I was
working on this project, and addressed each one.

1. Injection
We use 100% parameterized queries, and we apply strict
validation to all arguments of all shell scripts.

2. Broken Authentication
We have used Flask's session implementation,
we did not roll our own sessions.

3. Sensitive Data Exposure
We do not handle particularly sensitive data such as
cryptocurrency wallets or credit card information.

4. XML External Entities (XXE)
We do not parse XML.

5. Broken Access Control.
We have added the user's email address to all database
queries that we can. This email address comes from the 
session, so hopefully you can only ever get information
about YOUR account, and only if you are logged in.

6. Security Misconfiguration
We made sure that the application does not display error
messages to the user, we are not running Flask in
development mode, we are not running Flask as the root
user, the server it runs on is well secured and up to 
date, etc.

7. Cross-Site Scripting (XSS).
We apply strict validation to user inputs that will be 
represented on the page, whether they are path variables,
query parameters, form fields, etc. 

8. Insecure Deserialization.
We use the most up-to-date json parsing from the 
Python standard library.

9. Using Components with Known Vulnerabilities.
We did check the CVE lists for any known issues with the
versions of Flask and psycopg2 (database connector), 
requests, and various other packages that we are using.

10. Insufficient Logging & Monitoring.
We may have some room for improvement here, however,
verbose logging goes slightly against the "we don't 
collect any more data about you than we need to" mantra.

If you would like to take a peek at the code, it's
hosted on forge:

https://giit.cyberia.club/~forest/capsul-flask
\ No newline at end of file
diff --git a/site/blog/blog.xml b/site/blog/blog.xml
index bd7ed28..329d0cb 100644
--- a/site/blog/blog.xml
+++ b/site/blog/blog.xml
@@ -29,5 +29,13 @@
<pubDate>Tue, 25 Feb 2020 00:00:00 +0000</pubDate>
</item>

<item>
<title>20/05/2020: Capsul rollin' onward with Web Application</title>
<description>A necessary post</description>
<link>https://cyberia.club/blog/20200505-capsul-rollin-onward-with-web-application</link>
<guid isPermaLink="true">https://cyberia.club/blog/20200505-capsul-rollin-onward-with-web-application</guid>
<pubDate>Wed, 20 May 2020 00:00:00 +0000</pubDate>
</item>

</channel>
</rss>
-- 
2.20.1
I tacked on a link in the site/blog.txt file & merged this. Thanks!