Episode 39 of The SitePoint Podcast is now available! This week, Kevin Yank (@sentience) catches up with Twitter API Lead Alex Payne @al3x at Edge of the Web 2009 to discuss current trends in the technologies we use to build the Web, and where they may be leading.
Listen in your Browser
Play this episode directly in your browser! Just click the orange “play” button below:
A complete transcript of the interviews is provided below.
Download this Episode
You can also download this episode as a standalone MP3 file. Here’s the link:
- SitePoint Podcast #39: Web Futures with Alex Payne (MP3, 18.5MB)
Subscribe to the Podcast
The SitePoint Podcast is on iTunes! Add the SitePoint Podcast to your iTunes player. Or, if you don’t use iTunes, you can subscribe to the feed directly.
Interview Transcript
Kevin: December 4th, 2009. Twitter’s API Lead shares his thoughts on where the technologies we use to build the Web are leading. This is the SitePoint Podcast #39: Web Futures with Alex Payne.
Kevin: This is Kevin Yank for the SitePoint podcast in Perth for the Edge of the Web conference and I’m here with Alex Payne, one of the developers, in fact, the lead developer behind the Twitter API. Is that right?
Alex: Yep, that’s me.
Kevin: I guess a lot of people, when they get to meet you, they immediately think about the Twitter API and all the things that’s made possible; from your perspective, what does Twitter’s API mean to Twitter?
Alex: For us, it’s meant the flexibility to focus on what we do best or at least what we’d like to do best. It’s meant that over the last couple of years when we’ve been focusing on scaling the service and trying to add really high value features, things like lists that we just rolled out, that people have been asking for ages that we can dive in on the user experience and the performance of features like that and things that are a little bit more narrow or a little bit more nichey, developers can make use of the API and kind of build up those tools for themselves and for their users.
Kevin: So it enables the users to, in a sense, tell you by implementing things how it is they want to use the service.
Alex: Yes, definitely.
Kevin: So your talk was more general. It was really about where we’ve come from in web technology and where we may be going. One of the things you mentioned that really peeked my interest was that more and more web services like Twitter can and should explore an API-first methodology, that they can build their API and then “eat their own dog food,” you said, in building their web user interface. Would you say this is the approach that was taken at Twitter and can you think of any other examples where that sort of approach has worked?
Alex: Well, I’ve heard from folks on the Flickr team that that’s kind of how they’ve approached their architecture. It’s not the approach we took at Twitter. Our API kind of fell out of our user-facing application and if we could kind of do it all over again, I think I would rather have started from the API first in part because that’s been our primary source of traffic and in part because dealing with the issues of an API, the design issues, the performance issues kind of really shores up your thinking about your system.
Kevin: I’m a tech guy, what would you say is the technology mix at Twitter? I’m thinking not just the web languages you guys use, but I mean, are you all Mac people at Twitter?
Alex: For the most part, yeah. I think everyone now is running on Apple hardware, all of our engineers. We have a couple of people who boot into Linux day to day but for the most part, we’re all Mac folks.
Kevin: Cool. And you mentioned in your talk basically that at one point you were working on a pre-PowerPC Mac, so you must have been through the bad days of being a Mac user.
Alex: Yeah. I think that was a Performa 575 years and years ago and shortly thereafter, I had gotten my first Wintel hardware and went off into the land of Linux until OS X improved enough that people could actually use it.
Kevin: Okay, so you did move back and forth?
Alex: Yeah, and I’m glad I did. I mean I think like a lot of Mac folks, we all had that sort of crisis of faith for a few years and went off and learned Linux and it ended up being sort of a career asset, you know?
Kevin: Have your reasons for using the Mac today, are they different than the reasons you were using a Mac back then?
Alex: I mean back then it happened to be what was around the house. Today it’s more that nobody else has that kind of out-of-the-box, everything works experience and nobody else produces hardware of that quality which is just baffling. Like some days, I would actually prefer to be running on Linux so I could do weird things like run a tiling window manager but looking around at non-Apple hardware, nobody’s making good stuff even if you want to spend $2,500, it’s just really hard to get a nice machine.
Kevin: Yeah, one of the last Mac hold-outs at the SitePoint office, ironically, was our designer and recently, he just went out looking for a laptop and was not at all planning to get a Mac but he couldn’t find a good quality PC laptop and ended up buying a Mac for that reason.
Alex: Yeah, it’s really, really surprising. I don’t know what the Dells and Sonys of the world are thinking but it’s very odd.
Kevin: So one of the distinctions you drew when talking about the various web programming languages that are out there in the mix today is you distinguished between languages that are general purpose languages that are being used on the Web versus languages, like PHP and JavaScript, that were conceived for the Web, and you mentioned that there’s a tendency for these made-for-the-Web languages to have a shorter life span. Why do you think that is?
Alex: Well, not necessarily a shorter life span but at least that they tend to be kind of roundly criticized. People have complained for years about PHP encouraging bad development habits and being a very haphazardly designed language and people have complained about JavaScript having this object model that’s very different than most of the rest of the programming language world and having oddities and that sort of thing.
Kevin: Speaking of oddities, we’ve got some noise happening here, but go on.
Alex: So I think the part of the reason for that is the folks who’ve worked on more general purpose languages have had to satisfy users in a variety of different fields. They have academic users, scientific users, people building desktop software, people building server site software, and I think it kind of rounds a language out, it forces like Guido van Rossum’s or the Matz’s of the world to kind of expand beyond their initial concept of the language and figure out how do we improve the API’s, how do we improve the core libraries, that kind of thing. If a language is targeted at the Web and it has this more narrow audience, I think you never see that kind of cross-communication between programmers and different disciplines. Like PHP got a lot better when you could first start using PHP for client side or, sorry, for like command line scripting and that sort of thing. Suddenly, the language felt a little bit more robust and now I think JavaScript is going through that process as well now that people are using things like the Rhino JavaScript virtual machine for doing server side coding. It seems like they’re rethinking the language and kind of improving it.
Kevin: So the more use cases the stronger the language tends to get.
Alex: Seems that way, yeah.
Kevin: On that point, and this is something you mentioned also in your talk that on the Web it seems like the best technology isn’t always the one that wins, and this is something that Mark Pilgrim brought up this week when looking at old list posts about the <img> tag and how that ended up being the chosen option for HTML. Is this a problem on the Web that things that are evolved within just the context of the Web aren’t always the technically superior solutions and is this getting worse or better?
Alex: I don’t know. I go back and forth on this. I mean it’s kind of age old debate in technology, it goes back to Guy Steele “worse is better” kind of stuff. I think where it’s useful on the Web is that we’re going through such a constant period of evolution, kind of sloughing off the old skin, that if the technologies we’re using today are a little bit flawed, it’s easier to make the migration to the next set of technologies. If we had perfect solutions or solutions that even covered like the 80% or 90% case, it would be much harder every three or four years to kind of reinvent the Web, reinvent how we do all this.
Kevin: Another issue you touched on in your talk was that we’re moving into this age where rather than learning languages, more and more people are learning frameworks built on top of languages and the cost that comes with that. How do you, as a programmer, personally guard yourself against those costs? How do you hedge your bets? What’s your approach to that?
Alex: What I’ve tried to do over the last several years is, I mean, I’ve spent a lot of my day job working in Ruby on Rails, which has been, for the most part, incredibly productive, we’ve just gotten a ton of work done in that framework at Twitter, but outside, I’ve spent a lot of time learning about other programming languages, trying to look at problems outside of the web space, and particularly problems that are a little bit lower level so that it kind of balances out. You have that very high level framework-orientated thinking where you’re just focused on “how do I accomplish these business goals for the Web in as little time as possible?” It’s one way of thinking, but then saying how am I going to implement an entire programming language or even looking at problems in the security space that I used to work in. It just gives you a very different perspective.
Kevin: You as an API developer, a creator of an API, must have a unique perspective because people with so many different tools in so many different languages are consuming that API. So you have to build something that works naturally whether the consuming party is a Rails developer, a PHP developer?
Alex: Well, I mean, thankfully, since Twitter is more or less a RESTful API, there are conventions in most languages for dealing with RESTful web services, so we kind of get the benefit of all of that thinking.
Kevin: So you just finished up a book for O’Reilly about the Scala language, sell me on the Scala language.
Alex: Okay, so Scala in a nutshell: the advantages are that you’ve got object-oriented programming that everyone already knows and is comfortable with, you’ve got functional programming, which a lot of people have had a little bit of exposure to, but generally, they found that the syntax or the libraries in the functional language that you learned be a Lisp or Occam or Haskell or whatever were a little bit lacking. And the nice thing about Scala is that because it builds on the JVM and is completely interoperable with all those libraries, you can immediately take advantage of 10, 12 years of Java history. So we might think of Java as maybe not the sexiest language anymore but there’s a great collection of libraries that have been written over the years, people have been solving really hard problems in Java for a decade, and in Scala you can work in a beautiful language that looks a little bit more like Ruby or Python or something that more developers are comfortable in but you still get the benefit of all the JVM goodness.
Kevin: It sounds almost like it has some of the strings that made PHP such a success in that PHP, when it became popular was when they added the object-oriented features but they were optional. In the same way, Scala gives the functional programming stuff but you don’t have to leave behind all your skills and history.
Alex: Yeah, exactly. I mean, I know a lot of people are interested in Erlang as kind of a first functional language, and Erlang is great and it’s particularly great for a kind of network service problems but it’s much more restrictive. Everything has to be immutable. So you assign a variable once, you can’t assign it again. It’s a very different way of thinking than we’re used to in a lot of OOP languages and it’s very specific about having this kind of odd Prolog-derived syntax and all the rest of it. With Scala, it’s a little bit more forgiving. The language leaves a little bit more up to the programmer about whether you want to work in an object paradigm or functional paradigm, whether you want to solve concurrency problems with the more Erlang-style actor model or whether you want to go back to traditional Java threading and that kind of thing.
Kevin: And because it’s on the JVM you have the freedom if you can build a particular module in Java and then use something like Scala to access it.
Alex: Yeah, exactly. Like if you’re working in a Java stack like Hadoop, you’ve got all the power of Hadoop available to you. You can just call into it seamlessly from Scala. Things like that are a big time saver.
Kevin: So, as you said, Scala is it runs on the JVM, what are your thoughts on the JVM as a platform? Clearly, it’s got many years behind it and it’s a solid, powerful foundation, but I don’t know, I’m not as confident in Sun as I used to be. Do you feel like the JVM has freed itself enough of Sun that if worst comes to worst and Sun was to collapse, would the JVM leave on in a satisfactory way?
Alex: I think so. I mean I was at the JVM Language Summit a few months back and I’m not a JVM expert by any stretch but it seems like between Sun and IBM and all the rest of the folks who are dependent on the Java Ecosystem that someone will pick up the JVM mantle. There’s too much good engineering in there and there’s now the OpenJDK project as well as other various JVM distributions. So I think there’s quite a lot of choice.
Kevin: In addition to functional languages, one of the other, sort of “stabs in the dark” predictions about the future you made was that service-oriented architectures are going to come into their own, the idea that we’re going to become comfortable enough building reusable components of software for the Web that we will start to be able to plug them together in consistent and standard ways. It seems this is just like functional languages we’ve always heard, “Oh, next year they’re really going to catch on.” These ideas of clean software components particularly in the Web, it always seems to be about one year away and then just as things settle down, the hand coders kick in with something new and exciting. For a while, their WYSIWYG seemed like it was almost going to be a reasonable way to work and then suddenly a CSS layout caught on and no, no, back to hand coding.
Alex: Yeah.
Kevin: Is there another wave of hand coding craziness coming that’s going to wash this SOA away?
Alex: It’s entirely possible, yeah. In fact, I think in some ways, the JVM language thing might spur that on a little bit. I’ve seen more and more of the programmers that I follow on Twitter pick up things like Clojure and people are building these micro-web frameworks in Clojure that kind of closely mirror what people have been doing in the Ruby community for the last couple of years and the whole approach is very like every project is very lightweight, very bespoke, you just sort of put this thin veneer on top of other libraries and hack something together and Lisp has sort of always encouraged that, that very, more sort of agile like playful development methodology rather than this big enterprisey SOA kind of thing. So if more and more programmers are jumping on that kind of bandwagon, then I think you’re absolutely right. We may see another wave “well, let’s just hack it together until it works and then move on to the next interesting problem.” That’s always a fundamental part of the hacker culture, right?
Kevin: I’ve one last question and it has to do with the idea that as web services, these services are being built and it seems like, especially in the case of things like Twitter, the non-web user interfaces— As you said, you found it easier to explain to your mom I think, how Twitter worked when it was a desktop client that looked like instant messaging rather than a web page. Are we ever going to get to the point where a web service will not need to have a web UI and it’s a pure web service and that the primary user interface isn’t in a web browser? I know Cameron Adams who’s one of the designers behind Google Wave was speaking about a month ago at Web Directions South and he said the most interesting challenge he had as the front end designer for the Wave web client was that he needed to make something that was good enough that would attract people to the Wave service but not so good that it would quash that market for third party non-web clients.
Can we get to the point where the web browser isn’t the most important thing for a web service?
Alex: Yeah, I certainly hope so. It seems like more and more of the destinations on the Web really have nothing to do with the Web. The Web is just incidentally the delivery vehicle but we’re not longer delivering hypertext. Like if you think about the Hulu video service that is quite popular in the US, I mean, it’s replacing TV for a whole demographic. That really has nothing to do with being on the Web. I mean you can kind of click around from one show to another but they very quickly delivered, I believe, an Adobe AIR based application maybe a few months, maybe a year after they had launched and for most of my friends, that’s their TV now. It’s running this Adobe AIR application that talks to Hulu and streams video. The web site is basically just you get registered on that and then they point you at the Adobe AIR app and that’s really the better way to interact with the service.
Kevin: I have to say here in Australia where we don’t get Hulu we’re all insanely jealous.
Alex: Do you guys have Spotify though?
Kevin: No, no, we don’t even have that here.
Alex: You don’t have Spotify? Yeah, so in the US it’s like the tradeoff is we have Hulu but we don’t have Spotify, but you guys are getting screwed.
Kevin: We get nothing.
Alex: But yeah, I think that’s absolutely going to be more of a trend and honestly, looking down the road at solving problems beyond Twitter, I’ve been working in the browser for the majority of my career and it’ll be really, really nice to get out of that model a little bit. I like web services, I like working on API’s and that sort of thing but it seems like mobile, rich clients on the desktop, all these kinds of stuff is exciting. There’s a lot to learn there.
Kevin: Alex is @al3x on Twitter and thanks for taking the time, Alex.
Alex: Thanks so much.
Kevin: And thanks for listening to the SitePoint Podcast. If you have any thoughts or questions about today’s interview, please do get in touch.
You can find SitePoint on Twitter @sitepointdotcom, and you can find me on Twitter @sentience.
Visit sitepoint.com/podcast to leave comments on this show and to subscribe to receive every show automatically. We’ll be back next week with another news and commentary show with our usual panel of experts.
This episode of the SitePoint Podcast was produced by Karn Broad and I’m Kevin Yank. Bye for now!
Theme music by Mike Mella.
Thanks for listening! Feel free to let us know how we’re doing, or to continue the discussion, using the comments field below.
Related posts:
Comments