SICP Initial Thoughts

After installing Racket I opened my browser to MIT's excellent book, Structure and Interpretation of Computer Programs. I have been through a four year engineering course, but never in all my time have I come across a textbook quite as impressive as this. I only had to get as far as the foreword to be amazed by the level of thought and clarity of writing observed in this text.

It opens with a strong and declarative first paragraph.

Educators, generals, dieticians, psychologists, and parents program. Armies, students, and some societies are programmed. An assault on large problems employs a succession of programs, most of which spring into existence en route. These programs are rife with issues that appear to be particular to the problem at hand. To appreciate programming as an intellectual activity in its own right you must turn to computer programming; you must read and write computer programs -- many of them. It doesn't matter much what the programs are about or what applications they serve. What does matter is how well they perform and how smoothly they fit with other programs in the creation of still greater programs. The programmer must seek both perfection of part and adequacy of collection.

To me, this first paragraph for the entire book gets to the real heart of programming. What it is, why we do it, and how. It invokes the mystical and powerful nature of programming that has entrapped many a young technologist.  It describes the difficulties faced on the road to learning this dark art; and hints that the road will be a long one. Programming is an art; perfected through practice and discipline.

Every computer program is a model, hatched in the mind, of a real or mental process. These processes, arising from human experience and thought, are huge in number, intricate in detail, and at any time only partially understood. They are modeled to our permanent satisfaction rarely by our computer programs. Thus even though our programs are carefully handcrafted discrete collections of symbols, mosaics of interlocking functions, they continually evolve: we change them as our perception of the model deepens, enlarges, generalizes until the model ultimately attains a metastable place within still another model with which we struggle. The source of the exhilaration associated with computer programming is the continual unfolding within the mind and on the computer of mechanisms expressed as programs and the explosion of perception they generate. If art interprets our dreams, the computer executes them in the guise of programs!

This is such a beautiful description of our art form. It lays plain that the real product of a programmers' work is not code; it is a model. This model dictates how we interface between the real world and the logic of a computer. This model may be but a part of a larger model, an actor in a greater world. As programmers we wrestle with these models; we seek to understand and optimise the model, until we can finally fit it within the constraints of a computer program.

Unfortunately, as programs get large and complicated, as they almost always do, the adequacy, consistency, and correctness of the specifications themselves become open to doubt, so that complete formal arguments of correctness seldom accompany large programs. Since large programs grow from small ones, it is crucial that we develop an arsenal of standard program structures of whose correctness we have become sure -- we call them idioms -- and learn to combine them into larger structures using organizational techniques of proven value.

Now the writer speaks about the content of this book, and its purpose. Computer programs always start small. Large computer programs are often made of many smaller programs. Unfortunately, as larger programs form, it becomes harder to manage the agreement of specifications and implementations. This is where computer science can help. By forming standard techniques in the form of idioms and algorithms, we can ensure consistency and quality in the approach taken no matter what problem is at hand.

Next, the language used in this text is considered.

Pascal is for building pyramids -- imposing, breathtaking, static structures built by armies pushing heavy blocks into place. Lisp is for building organisms -- imposing, breathtaking, dynamic structures built by squads fitting fluctuating myriads of simpler organisms into place. The organizing principles used are the same in both cases, except for one extraordinarily important difference: The discretionary exportable functionality entrusted to the individual Lisp programmer is more than an order of magnitude greater than that to be found within Pascal enterprises.

I think this is actually a reasonable approximation of the difference between the widespread modern languages such as C# and Java in comparison to some more academic or experimental languages. The former often strive to build structures and machines that may be very static and uninteresting in nature, but will stand the test of time. The latter type of languages seem to be very much more dynamic and unstable in nature, often trying new and outlandish ideas in particular areas of the language for a potentially positive gain. The writer finishes by adding:

As a result the pyramid must stand unchanged for a millennium; the organism must evolve or perish.

This is also a great description of modern businesses; the current hive of innovation in silicon valley is all about this "fail fast, evolve or perish" type of environment; meanwhile many much larger companies (and industries) that have long stood the test of time are starting to crumble in the face of such opposition.

To me, this book has already stirred deep thoughts on the nature of computer programming, and of the technology industry at large, and I have yet to write a line of code. It will be interesting to see whether the rest of the book is as enlightening.

All quotes are sourced from the Foreword of the book Structure and Interpretation of Computer Programs by Alan J. Perlis.

A Journey into Functional Programming/Lisp

I've officially been a "Developer" for about 2 years now, but still I can't shake this feeling that ever since I left university my skills have been stagnating. Maybe it's imposter syndrome or something; I just don't feel like the reality of the working environment has lived up to the ideals I conjured up while I was still undergoing tertiary education, or that I have lived up to what I thought I was capable of. Anyway, driven by this feeling of insecurity in my current skillset, I've been doing a bit of research on Functional Programming (FP), and in particular the Lisp-like languages which seem geared mainly towards FP.

My interest in functional programming was spurred by this blog post which turned up on Hacker News. It doesn't completely explain FP, but it does a good job of summarizing a few key points which definitely got me interested enough in Functional Programming to try and learn a little more. After searching around the web for information on Functional Programming there seemed to be at least 3 main languages used for functional programming; Haskell, Erlang, and Lisp. Yes, I know this glosses over many other Functional languages. These were the ones that I identified as the most widely used functional languages. That sentence seems to be a bit of an oxymoron: my research also revealed that Functional languages are only used sporadically outside of a few specific domains such as academia (Haskell) or telecommunications (Erlang).

I settled on Lisp for multiple reasons:

• Lisp is one of the oldest high-level Programming languages around, only rivalled by Fortran. It's been around for 50 years, and there's a good chance it'll still be around for another 50.
• Lisp has a range of different implementations which apply different tweaks and changes to the language, but all keep most of the same underlying syntax. This means Lisp is actually an ecosystem of many different languages, from Common Lisp, to Scheme (a whole other sub-category of Lisp-like languages), to Clojure.
• The language itself isn't that much of a jump from imperative programming. It is actually simpler than some mainstream programming languages; most Lisps break down into 3 main types of syntactical elements: lists, labels and atoms/values. It made sense immediately when I saw examples of it. I couldn't say the same for the first Haskell or Erlang code I looked at.
• There's some great material available from MIT on learning Lisp, since they used to use it for their introductory computer programming courses, as well as artificial intelligence work.

As I mentioned, Lisp is technically a collection of languages which have been derived from the original Lisp implementation over time. Scheme is a sub-category of these derivatives that focuses on a very small and stable core that can be easily extended. This makes it an excellent choice for an educational setting. In my case I chose to go with one of the (currently) most popular Scheme implementations: Racket (formerly PLT Scheme). It comes with an Excellent REPL (Read-Eval-Print-Loop) environment and plenty of built-in functionality so programmers can get up-and-running very quickly.

Now my plan is to work my way through How to Design Programs and Structure and Implementation of Computer Programs; hopefully I'll learn something in the process. Either way, I'm sure it will be an interesting journey into the world of functional programming and Lisp-like languages. Hopefully this will prepare me for a closer look at more advanced languages such as Haskell or Erlang in the future.

Free market ticket sales, the only real way to kill scalpers?

It seems to be all over the news at the moment; new legislation and ticket systems have done nothing to abate scalping of tickets for concerts and events in New Zealand and all over the world.

Scalpers depend on the supply/demand characteristics of a market being tipped far in favour of demand. What if the demand is removed? Ticket vendors and event organisers have tried to implement this with threats that tickets found to be onsold will not be honoured. The logic behind this is that if people don't buy tickets from scalpers, there will be no market for the scalpers to exploit. The failure on the part of the organisers and vendors is that this implementation of that logic is fatally flawed; good honest people still want to buy tickets; and because there is a finite supply of said tickets, they have to buy from the only source available: scalpers. This has been somewhat successful in decreasing the demand in the scalping market, but it has also meant that some people have had to make the conscious decision not to buy tickets for an event that they want to go to.

My theory is that the only way to truly reduce the demand for scalped tickets to zero (or anywhere near it) is to allow market demand to dictate ticket prices. Yes, I can hear the horrified screams of bands and music fans everywhere as I say this. What do you mean we should have to pay hundreds of dollars to go to the concert by that band I love?!?! WTFBBQ?!?!

Sure, ticket prices would probably skyrocket immediately. But consider this; instead of the hard-earned money you spend on tickets going to scalpers (if you're unlucky enough to not get in early), the money goes to the promoter, the venue, and of course the performer. For big name bands with huge fan bases this would be a massive win in terms of revenue from concerts. But what about the social implications?

Many bands are known to impose limits on ticket prices on purpose, just so that they can allow their most loyal fans and followers to attend concerts. Allowing only the richest fans to attend a concert will mean the overall culture suffers, right? Suddenly who can go and who cannot is dictated purely by how much money that person is willing to pay. Is this fair? That could be an entire other discussion in itself, and I don't particularly want to get into it here.

My thoughts are less on the implications of this on the followers and attendees of events, and more on the effect it will have for scalpers and bands, and how such a system could be implemented to maximise success for both bands and fans. I want to see less money going to scalpers and more money going to the performers and musicians that people are paying to see.

So how would we implement such a system? The only way scalpers can be defeated completely is if there is no market for the tickets they buy. So how can we do this?

• Decrease the demand for second-hand tickets - this is the approach currently being tried by governments and organisers.
• Increase the supply of tickets - this simply can't be done for most shows, especially if you already have the biggest venue in town.

Obviously, the only real option is to reduce the demand for scalped tickets. How can we do this? The approach already being taken is to reduce the demand by increasing the risk to potential buyers. This results in a loosing situation for buyers who just want to see their favourite band when their tickets are invalidated or they are barred from entering a concert.

What if ticket sales are turned into a priority list, ordered by the amount paid, which is completely fluid until tickets are printed at some time before the concert? Suddenly legitimate buyers can simply outbid scalpers. If scalpers try and outbid the legitimate buyers, they will find themselves with tickets on the night of the concert that no one wants to pay more for (because those people have already outbid the scalpers). Of course this system would not eliminate scalpers altogether; there would still be a market for scalped tickets among latecomers who wanted to buy tickets at the last minute. The implementation of such a system would be complex because of all the stakeholders involved. And as I have said, there are significant social issues and business relationships that would have to be sorted through before such a system could be put in place. Not impossible, just the sort of operation that would be suited to a committed and skilled entrepreneur.

The other option of course is to put up with scalpers, and accept them as a natural evolution of the supply/demand market. Either way, people will always have to pay market rates for tickets, whether that market is set by the organizers or the scalpers. There is no escaping the supply/demand curve it seems. That may sound cruel, but it seems that it is the honest truth of selling such limited quantity goods. 

I'm only an engineer, not a marketing or sales person. So by all means, feel free to discuss any flaws or shortcomings in the theories and opinions I have expressed.