25 August 2016

I am happy to welcome Marcin Kulik, a software craftsman from Krakow, creator of Asciinema terminal session recorder. The interview is a part of series of articles focused on services created by Krakow’s tech scene. My intention is to present the original works of programmers who run extraordinary projects.

Marcin Kulik: Maybe it's because of the fact that I didn’t integrate myself with Kraków's start-up community too much. Furthermore, I never had really strong need to commercialise the product. You lose the freedom when project gains the importance and other people are involved in it’s development. Extra negative results I wanted to avoid are stress and expectations of investors who invest the money in it. At this point of development, it is a very free and easy relationship. There are no expectations nor demands for specific additions/features. It would probably change if my product became a commodity. Currently, Asciinema is my greatest hobby and brings me a lot of joy. For example, I can rewrite a part of the code because I want to learn a new language. Moreover, I know the problem domain well so I can try a new language/tech if I want it. And no one can tell me that I am wasting my time for unnecessary things. Although, users of open source community such as GitHub often feel like customers and they request for features/fixes as they would paid for it. Still, it's not the same level of obligations than when you have an investor.

What Asciinema actually is? From my perspective, this is both a player and recorder of terminal session. And online service like youtube with all recordings shared.

Asciinema is a multi-project: technically, there are several independent parts and their main goal is painless, simple recording of the work done in the terminal and which can be shared later on on the internet. One of the parts is a terminal recorder which works in the terminal and is available as just another shell command. From the moment of activation, it records all of the work that happens in your terminal window. The other parts are the website hosting the recordings and the web player used on the site.

asciinema.org

How the idea of Asciinema came to your mind? How it started?

Probably in 2011, when I had thought for the first time about creating a new solution; there was an old UNIX tool called 'script' which was recording all operations of the terminal to a file and later it could be replayed in user terminal. I thought that this is a cool practical thing but it would be much better if I could show my recording to someone in the easier way. At that time users were able to record it, send it by email or messenger and then ask to open and play on the terminal. All of the things above were, in my opinion, too complicated so I decided that there must be an easier way and possibility to create the tool that automatically records, uploads and hosts recordings to play them directly online. It should also give the opportunity to share all recordings with links. And that's how all of it started. Part of the project is written in JavaScript (specifically ClojureScript) and it's on the Asciinema page where you can download it and to embed it.

A YouTube for nerds.

Yes, it is!

Liquid simulation

And who is the average user of Asciinema?

First of all, developers and administrators. And more specifically, these of them who use systems of UNIX origin, such as Linux, Mac or OS X. Vim devotees also can use it, i.e. they can run the Asciinema, then run Vim, work in it, end the session and then do a showcase. Therefore, the recipients can be very diverse. Internet users, often programmers, want to show their skills to a larger group of people. For example, in the shell, you can connect multiple commands by “piping” them together, which gives you lots of possibilities. Linux and Unix users like to brag about things like that.

I've noticed that Asciinema is being used also on educational services, such as rspec.info or as a 'know-how/ how/to', but also in tutorials on Docker preparation and programming...

Yes, there are many educational recordings on our website. Many users want to show that they can do cool things or maybe they want to teach someone a skill. I thought it would be awesome if they could record their work and then make it available to everyone on the web. Also, some sites embed Asciinema recordings of tutorials on how to use their products or tools.

Do you know any companies or other institutions which use your tool?

I don’t know if it’s still the case, but there was a time when Docker had embedded player of Asciinema in its official documentation. It was a tutorial about how to containerize an application written in Python. But I'd have to see if it’s still there. Also, one of the most popular Ruby test runners, RSpec, uses Asciinema on its homepage for mini tutorials, showing how to write tests in RSpec. Furthermore, several graduate students and university professors have written to me and congratulated me the idea. They told me that they record all of their presentations in Asciinema and show the recordings to their students during lectures. I got a pretty good feedback from them! Also, other organisations, such as one of the Austrian banks, want to have an internal instance of the Asciinema site available within the company. They love Asciinema.

Behaviour Driven Development for Ruby on rspec.info

On your website Ican see a lot of fan experiments, i.e. how todos, videos converted to ascii, demos, fractals in a text mode, games, code showcased etc.

Yes, on asciinema.org you can view public recordings (or private ones with a secret link) and there is also a special section with the most interesting, colourful and catchy recordings. From time to time, people upload really funny things. Once, a user has dropped a little game of a ski jumper. DOS-like game in 2D where you can manoeuvre your skis and gain points. Someone built this game using terminal drawn characters (letters, digits and other special Unicode characters like lines, blocks etc), not pixels, and it is still available on the website. There are also other games and demos. We have Space Invaders-like shooter, and we even have animated fractals. However, these catchy ones are exceptions because 95% of the recordings are more traditional: commands, code writing, programs launching, etc. Therefore, the main objective is to document and show some useful work to viewers.

Wolfenstein in 600 lines of code

Let's talk about more technical aspects of Asciinema. The great characteristic of the project is the fact that videos take up very little space (50-100kB).

Yes, it's because when you’re typing in a terminal, any pause in typing doesn’t produce any visible output and thus it doesn’t increase the size of the recording. It works like that: the recorder plugs in into your terminal and captures all of the text that is printed. While capturing it still forwards all the text to the display (so your terminal window can show it to you). It's a kind of a middleman who plugs and captures quietly. This means that if you stop for a while to think about what command to type next, nothing happens. There is no text printed, so nothing new is recorded. This approach really helps to limit the size of a recorded file. Only when someone types something, or when the application that is already running in your shell shows something (command completion, progress indicator etc), then these events are logged. For example, if you started recording today and came back after a year to finish the recording session, the file size would be a few dozen bytes, mostly used by metadata, such as the height or width of the terminal.

It looks like it works in the same way in OS X systems.

Yes, it works on the basis of a pseudo terminal, a feature of OS X systems, but unfortunately not for a Windows system. There was an attempt of the Internet user He or she tried to hack it and with a constant framerate tried to intercept a windows terminal screen in diff between one frame and the other. Overall, it works after a fashion. This is not something I want to include to the main project. Although, there is a hope as Windows 10 demonstrated bash, through which most of the terminal Ubuntu programs can be run under Windows 10. I saw that a Twitter user reported information about running the shell at Asciinema Windows 10 without any modification, and the recording is available on Asciinema.org. The problem might be solved soon.

So, we are talking about the recorder application that runs on the command line, now I record what I write and then push it to the internet to share it or embed in any site. But with no streaming possibility. I've heard that you're working on it.

Yes! Asciinema is the project I develop in my free (and thus limited) time, but I plan to take care of this. Streaming is definitely one of the features I want to add soon.

I see. And in what format you are currently recording in and how do you store the content? how does it evolve over time?

At the beginning, I wanted to use the old Unix script command which could do all the recording for me. The work of the recorder would be to use script command under the hood and then upload that resulting file to the website to be viewed. It turned out that this tool wasn't a good solution because in OS X it doesn’t record timing information. So, it produces a text file containing everything that was printed to the terminal but it cannot be used to do frame-by-frame animation. Moreover, script doesn’t store terminal dimensions, and it’s very important to have this information during playback later. Ttyrec, another tool I considered, also didn’t capture width/height of the terminal. So I decided to implement it myself. Let me explain why knowing original terminal dimensions is important during the playback. The sequences of characters displayed in terminals are not only pure text, but they also include controlling bytes that change the colour of the text or move the cursor around. Moving the cursor can be relative or absolute. For example, you may move the cursor to the last column at the end of the line by printing a special series of bytes. The meaning of the “last column” depends on the size of the terminal (its width). If I replay it in a terminal of different size then everything could have fallen apart and simply look bad. Now, as the Asciinema player runs in a browser and is written in JavaScript, it was a quite natural to use JSON file (with a set of documented attributes) to store the recording. Saying briefly, the recording is a sequence of tuples containing text to print and a delay between the provious and the current frame.

And which techniques are used by the recorder to read the size of the console?

The recorder uses the command tput available in all UNIX-like systems. You run it and it displays the current width and height of the terminal. When the recording session finishes tput is ran and its output is stored in recording’s metadata set.

Some of the components of Asciinema are, firstly, a service which hosts json and displays them to view and secondly - a player to be embed on website log. Could you tell more about it?

Gameplay of XorCurses based on Xor by Astral Software on ZX Spectrum

You can say the website is like YouTube because it hosts all recordings which can be viewed, shared (link) or embedded (widget). People are creating a presentation (for example HTML-based like using reveal.js), copy-pasting a snippet of JavaScript into a specific slide and then opening the presentation in the browser. This way they may do a live-looking demo of work in a terminal which in fact has been recorded the day before. It gives presenters a peace of mind and it has a better appearance for the audience.

Due to the fact that this is a recording of the text, it has an infinite quality when rendered in the browser. The terminal displayed by the player it is not graphics, but n-lines of text, just as in a regular terminal. That means you may copy-paste it from a browser. If you see something you would like to try in your own terminal, you can always copy-paste the relevant command. As you mentioned, the player itself has the ability to change the size of text, but regardless of this, the user can zoom the page in the browser and the font will remain sharp as a razor. Also, Asciinema player has some cool features, such as ability to change the playback speed. If you had a bad day and you were typing very slow during the recording session, then you can accelerate it during the playback.

And the player itself is written in the JavaScript ...

The first version of the player, including the control sequence parser, was written in JavaScript (with jQuery for rendering), in kind of a guerilla style. It used a trial and error approach to text parsing based on regexp matching, which was a really bad idea.Later on, the player (and its parser) had a lot of modifications and, finally, a year ago I rewrote it from scratch in ClojureScript. ClojureScript is the Clojure (modern LISP) to JavaScript compiler, so I write in Clojure, and in the result, I receive the .js file. It is one of my favourite languages. It was a nice opportunity to explore it on the domain I already knew.

Did you use or found any interesting conceptions while writing a player?

From the point of view of the regular user who uses the terminal and does not know how it works, it may seem that only plain text is ever printed to a terminal. In reality, the “printed” text often includes non-printable control characters, and each terminal emulator is a state machine that interprets all of this and decides if given byte should be displayed as a character (so we can see it) or should it change the internal state of the terminal. For example, a change of text colour does not happen magically. You print “\x1b[31m” to a terminal and it sets the text color for the following characters to red. Another interesting thing is moving a cursor - there are multiple ways of controlling the cursor position on the screen. The origins of this whole “virtual terminal” technology lay in the 60s, 70s and 80s and it t was being extended during these decades so its specification is rather broad. During the ClojureScript rewrite I decided to do it properly. About 10 years ago, Paul Williams has posted on his website an extensive diagram of a modern, compatible, virtual terminal state machine. Several other solutions like mosh (Mobile Shell) or libvte were implemented using Paul’s diagram. I did the same and implemented the complete state machine in ClojureScript. It was an awesome experience. As a result, I learned a lot, both about virtual terminals’ subtleties and Clojure’s data structures and functions operating on them.

Animated Julia Set

What are your future plans? We've already mentioned about streaming that theoretically would help in real-time sharing, for example, on Facebook, in Asciinema format. The lossless quality, due to the rather not demanding capacity which would enable the easy playback, for example, social media ...

The idea is to be able to stream your work live: start Asciinema streaming mode, get a link where this live stream will be available and which can be shared anywhere (chat, Twitter, Facebook) as a tool for web friendly as the rest of the-the entire project.

Is there a similar product you've heard about?

I saw some non-web tool allowing many people collaborating in the terminal. It works this way: I'm starting to work and share with you a token to which you can connect. It cannot be archived in a web and you cannot watch it live online. It is similar idea but caters to meets the different needs and use cases. Streaming is something that has been on my mind for some time and is 'coming soon'. I'm glad because I think it will open up new possibilities of this tool and attract another group of users who will be willing and able to show something live and on-line.

The product looks great, one of the well-received and made in Krakow!

Thank you! I'd like to add that while the project is already functional and meeting my original goal I have lots of ideas to make it even more awesome. One of the obvious ideas (and a frequently requested feature) is audio recording support. There are others, less obvious, like full text search in the recordings contents.

I'm glad I had the chance to meet you and talk about your project.

I am pleased, thank you for an interview and for the opportunity to present the project both from the outside and the "backstage".

Asciinema

Asciinema blog

Asciinema on Product Hunt

Asciinema on Hacker News

Asciinema on GitHub

Krakow, summer 2016

---