Friday, December 30, 2016

Thoughts on silversmithing and software engineering


"Yiannena, leader in arms, in piasters and in letters" is a famous Greek phrase that summarizes the status of the Greek city of Ioannina at the end of the 18th century and the beginning of the 19th. Indeed Ioannina, at that time a part of the Ottoman empire and under the leadership of Ali Pasha, were the wealthiest Greek city, had the most schools in the country and were a significant military force.

Ioannina gained a large part of the wealth by trading silver artifacts. The city gathered the greatest silversmiths and they were producing beautiful artifacts and eventually became known as the city of silversmiths. That tradition continued for many years and included many great craftsmen. One of the most famous was Sotiris Voulgaris who eventually migrated to Italy and founded the famous Bvlgari house.

Recently a new museum for silversmithing opened in Ioannina and i paid a visit today. It is housed on a beautifully renovated Ottoman building of the 18th century inside the castle of Ioannina. The museum consists of two levels.

The first level is the technological museum. It demonstrates all the techniques used by the craftsmen to construct the silverware. For each technique there is a showcase with tools, first matters and instructions about it. Next to the showcase is a screen that shows a video of a silversmith that demonstrates the technique to build an artifact. Almost all the techniques require great precision, timing and a very steady hand. Silversmithing is definitely not something you learn by learning a book. It requires years of practice. The most important techniques are: cupellation, hammering, raising, sand casting, filigree, savati and repoussage. Each of them produces a very different type of artifact.

The first level also contains information about the history and tradition of silversmithing in Ioannina, the bios of some great craftsmen and some illustrious clients of them. Also there are important informations about the social impacts. Greek readers should know that the phrases "ροδάνι η γλώσσα" and "ξετσίπωτη γυναίκα" are metaphors for some tools of the silversmiths.

The second level of the museum contains a rich exhibition of silver artifacts. It includes guns, swords, cartridge boxes, lucky charms, belt buckles, candelabras, bible covers, chalices, jewellery and much more. Each one of them is demonstrated in a proper showcase that allows viewing from all sides so that the visitor can fully admire both the piece and the craftsmanship. All these are accompanied by rich information on the social status of silverware through history.

In both floors there are touch screens with additional information and there is also a computer room with applications related to the museum like quizzes, encyclopedia and a drawing program with patterns from the exhibited artifacts.

In a nutshell the new museum is a jewel for Ioannina. It beautifully demonstrates a very important part of the city's history but also succeeds in hooking the visitor into the art of silversmithing. Add to this the beautiful building of stone and you have a very good reason for a visit.

My impressions from the visit should end here but alas i am a software engineer and can't avoid compare this art with mine.

Of course you can make many parallelisms like comparing software maintenance with cupellation (recycling silver), or compare sand casting with reusable libraries and frameworks. You can even go very far and compare filigree (a silver thread) with the flow of a program, or compare working on hot metal with testing on production. All these are valid but the main differences are two in my opinion.

The first one is visibility. A silver artifact is designed to be beautiful and breathtaking. There artifacts will be visible by many people who will admire both them and the skill of the craftsman. You buy silverware to impress. On the contrary, the best software is transparent. Users do not and should not care about the internals of the software but only if it gets the work done. They don't care about scaling they care only to find quickly what they search in the web. They don't care about clustering algorithms, they just want to find the best product to buy in the cheaper price. They don't care about containers they just want their machine to boot fast and never crash. They don't care about filesystems they just want to see the photos in their camera. The internals of software should concern only engineers not users. Although this is a very simple principle it is very often violated in practice. Engineers sometimes fail to properly evaluate the user's needs and they expose everything under the Trojan horse of configurability or as big tables of complex data or forms of colored buttons which is wrong. The correct way is to build it properly. Sometimes they focus too much on technology that software is impossible to be used by casual users.  But the worst thing is that most engineers think that something is good if there is a path to learn it, even if this path is difficult. This is very wrong. Computers and software should help users not vice versa and hence must be very easy to use. If a user has to learn a very different part of the software to do a slightly different job something is wrong. Good software is transparent.

The second big difference is craftsmanship. A silversmith masters his art and creates beautiful artifacts. His artifacts reflect good reputation to him and he gains glory and recognition. He may pass his knowledge to an apprentice if he desires. Well, not in software. We have learned that good software can only be build by teams not individual great programmers no matter how good they are. Also knowledge should be free and distributed quickly through communities. Although this is also obvious it is also very often violated in practice. Still some engineers believe in the legend of the great hacker/craftsman and they work that way. Undeniably they are very good in their work but they don't distribute their knowledge in the team and some times they don't want teammates, they prefer to work individually. But even in the case of a team the knowledge, the skills and the work are not evenly distributed and this causes great problems if the master fails to deliver. The situation is worse for communities as many good engineers don't share experiences and knowledge fearing that they may loose the master badge. Good software can be produced only by teams of uniformly skilled individuals very actively collaborating. Everything else that focuses on individuals and not on teams will fail in the long run.

Silverware is a luxury. A century ago people were buying silver artifacts to increase their social status. Today it is out of fashion as people prefer electronic gadgets and cars. However it may return in the future. But software is not a luxury, it is the engine of the digital age. Every day we rely more and more on software to do simple and complex things. We rely so much on it that it is crucial to be good and working. It is difficult to write such software but at least let not the myths of the craftsman and demonstration hinter us.

Friday, October 21, 2016

Impressions from Voxxed days Thessaloniki

For the first time, voxxed days was held in Thessaloniki, Greece. We, the community, were very excited about this but we were also anxious if it would comply with the high quality standards of the brand. And it certainly did! And it certainly met all our expectations!

I had a great day meeting colleagues and friends and new interesting people to talk with. The topics of discussion were of course the talks delivered during the conference but also and more important the state of the Greek IT which has been hit hard by the Greek economic crisis. But no matter how a discussion between developers starts, even for debt and IMF, it eventually becomes purely technical about microservices and designing for failure.

Thessaloniki is one of the most beautiful Greek cities and a great destination for food lovers, so no surprise that the venue, the food and the pastries were excellent.

Now the talks. There were three tracks: java/devops, big data/methodology and web. There were 2 common keynotes so you had to chose 5 out of 15 talks to attend. I decided to optimize for best practical value and i avoided talks about new technologies and future directions. I unashamedly cared only for the present.

The opening keynote was by Simon Ritter about the upcoming features of JDK9 and project jigsaw which adds modules to java. It was a detailed talk about the language changes and how they affect legacy and new code. Alas I am with those who believe that jigsaw is too late and it will be irrelevant. Developers have learned to live without modules using facilities provided by build tools, frameworks and application servers. I hope that the final release of the JDK9  next year will prove me wrong.

But while jigsaw value is debatable the value of HTTP 2 is not. In a very informative and lively talk Ole Michaelis presented the problems of HTTP 1.1, destroyed the myths about some web techniques that we accept as best practices but actually are limitations of the protocol and presented the next version of HTTP. It is supported on all major browsers and platforms and it can be used right away. I totally agree with him. Let's use better technologies to build a better web. Let's get rid of the asset pipeline that makes our lives bitter. A great talk, i enjoyed a lot.

Next was Stavros Kontopoulos who talked about data streaming engines and used Apache spark as a case study. He presented the design principles of such engines, guidelines for good design of applications that use them and showed examples with spark. It was the kind of the simple straightforward talk that helps you tidy up your concepts and views about the topic and rethink your approaches. Personally it helped me a lot to clarify that an application I maintain is actually a custom streaming engine as it has all the characteristics of them. Now it is obvious how it should evolve.

Most developers hate documentation. They believe the myth that if the code is well written there is no need for documentation. I adore documentation. I even prefer to write documentation than unit tests. Peter Hilton delivered a great talk about documentation and discussed all aspects of it: when to avoid it, when to write it, how to avoid it without harm for the software, how to write it well and many many more. The slides were by itself a beautiful document on documentation. Find them and use them as a manual. This was imho the best talk of the conference.

Now it is time to become technical. Ali Kheyrollahi discussed best practices to write microservices. These are in a nutshell: ActivityId propagator, retries and timeouts, io monitoring, circuit breaker and canary, health endpoints. Although the context was microservices they are applicable to general network programming. The essence of his talk was that distribution is about reliability and in distributed systems everything can fail, so design for failure from day 1. This is of course a big topic but the talk succeeded in making it's point clear.

Dimitris Andreadis from redhat presented the latest versions of wildfly and wildfly swarm. These are JEE application servers for the cloud. Plain wildfly is a classic server on steroids, very fast and extensible. Wildfly swarm is a modular version of wildfly that you can assemble and configure as you wish. It has very small footprint, and has all the functionality of wildfly. This is the version for cloud deployment. JEE is still strong although not as dominant as it used. I really liked the architecture of swarm. Note however that many of the decisions were made to bypass the lack of modules in java and swarm's design is a better foundation for modular apps than jigsaw. Nice project but personally i like dropwizard very much to change it.

The closing keynote was by Sandro Mancuso about career design and development. To be honest, in Greece this is frequently discussed between developers because the environment is generally toxic for good career development. There are good jobs, good companies, good engineers but no full path for career development. We have the dots but we cannot connect the line, to paraphrase a classic talk by Steve Jobs. Sandro gave us valuable advices but most importantly he transmitted positive energy to us. Thanks Sandro.

Voxxed days Thessaloniki finished with good news. Next year  there will be Voxxed days Athens and we hope it will be as good as this one. Greece is back to the map of good IT conferences.

Spyros

PS1 I am writing on a bus without wifi, so i can't provide links. You will find them at the web page of the event but first make a small stop to check the people who organized this. Greece is not back on the map of conferences by luck or accident. People worked hard for it and they deserve our appreciation.

PS2 Outside the venue there was a chess event. I bet nobody figured out that the guest of honor was the girls world champion under 18 which is Greek! Google for her, she will win the women world championship one day.

Tuesday, October 11, 2016

Impressions from DotGo 2016

The best thing about go is it's community. It has taken a language designed for systems programming and used it in many alternative ways. This boosted the development of the language and emphasized the consistency of go's design.

I attended dotgo last year and enjoyed it very much, so I came to this year's event with great expectations and I wasn't disappointed. The organization, the venue, the food and the happenings were excellent. The highlight of course was the giant French gopher which dominated the stage, and i also liked very much the idea of a professional illustrator that drew the speakers and the best moments of the conference. Unfortunately I am writing from a mobile device of limited capabilities and i can't post links, photos etc. You can find them at Twitter at the organizer's account @dotgoeu. They will also publish the videos from the talks, freely available to the public.

Now, the talks. I will present them not in the order delivered but rather grouped in three groups according to the major aspect of go they revolved around: language, tools, environments.

Language

These are the talks about how to write good idiomatic go.

The first one was from Dave Cheney and he talked about first class functions. He presented examples of functions that return functions and used them to simplify code and write more coherent APIs. This wasn't about functional programming. There is a duality between single method interfaces and functions and go's simple design makes it very easy and straightforward to alternate between these two, something that is very difficult in other languages. In go we get benefit from both sides of the duality.

The next talk was from Damian Gryski who talked about caching and memory layouts in modern architecture. He tried to simplify and present advanced performance topics and he did a great job. His advice: measure first, optimize later.

Peter Szilagyi was next and he discussed immutability and concurrency. Go has no support for immutable objects, so to build his object graph he needed some scaffolding. The resulting code was simple to read, understand and extend. It favors go's simplicity but raises the question if the language needs support for immutability.

Katrina Owen talked about a technique for refactoring that she calls "noop noop noop interface maneuver" and she used it on a buggy piece of code to make it more readable and testable. Basically the technique extracts pieces of code to small interfaces and assembles them in helper types. This can be done easily and fast in go because the types are open and there are no type hierarchies that you must be careful not to break or bloat.

The final talk this group was by Robert Griesemer, one of go's creators. He gave a talk about software design and prototyping. As a working use case, he extended go to support two dimensional slices and showed how good is go at rapid prototyping. A great talk that emphasized the cultural difference between hackers and engineers.

Tools

A good language should support good tools. Go does this excellently.

Rhys Hiltner presented practical examples of using the go execution tracer. This is a tool that gathers execution events, for example a goroutine starts, and presents them in a compact form to the user. It can be used alongside a profiler to check how a large program behaves. Rhys presented examples of application bugs, false alarms and bugs in the go runtime. That is the talk that emphasized the design consistency of go. You get an excellent tool from the standard library and you can use it in a uniform way to investigate your application and the go runtime all in once. Compare with other platforms that rise barriers between your code and the platform.

Next was Matthew Holt who talked about ACME, the automatic certificate management environment and how easy it is to use from go. He argued that we should encrypt our apps since now it it very easy especially in go. I am not a security expert to have an opinion on this but the talk received many kudos from people involved with the subtleties of TLS.

Environments

Go was designed to run in a networked environment but what exactly is a node of such an environment? Is it just a traditional computer?

Dmitri Shuralyov maintains gopherjs, a go to javascript translator, and he believes that a browser can be a very friendly environment for go programs. He run ivy, Rob Pike's apl interpreter, in the browser, used chrome to stream http content written with net/http and as this wasn't enough he demonstrated an example of GL written in go, running in the browser. A great talk. The point is that it makes you rethink the basic interfaces of the standard library and how they map to the browser environment. What is an io.Reader in chrome and how well the fetch API maps to net/http and vice versa. After you define these building blocks the code is the same, both for the browser and the desktop computer.

Go can also run very well in mobile devices. Elias Naur talked about the latest version of gomobile and the features that allow you to write go and create bindings for Java/Android and Swift/iOS. The percentage of go code in a mobile app increases but we are not to 100℅ yet. Daring explorers can program Android in Go straight away.

Next was Kelsey Hightower, of kubernetes fame, who talked about why static binaries that bundle application code and the platform runtime are great for deployment. He demonstrated the concept using kubernetes and increased the size of the cluster for each run. Go binaries don't even need a container like docker, although it is highly recommended but for other reasons not for the sake of deployment. It was a very nice and lively talk about the go feature we all loved from the first days of the language.

Brad Rydzewski talked about plugin architectures for go, or in his words "designing for the unexpected". When you design a system and you release it to public, you can't predict the changes. Sometimes a plugin architecture is more suitable for projects that customization needs are highly anticipated. This is a case where our beloved single self contained binary is not always the best choice. There are alternatives like custom binaries or RPC to a process running the plugin or RPC to a container running the plugin. Brad discussed all these and the new architecture of his CI tool that incorporates these ideas. Clearly this is an open topic and I would like to see a solution in the standard library.

Conclusion

I really enjoyed this year's dotgo and I left with new ideas, new images and new friends. The go community has the best balance between going forth and stay relevant. We want to advance but we also want to stay relevant. It is a great feeling to be a part of such an active and prospering community.

See you next year at dotgo 2017