Toru Yamaguchi: OpenID 2.0

An introduction to OpenID, how it works, and how it can be used with Perl.

Yuval Kogman: Moose

An overview of Moose, the much-talked-about new approach to object-oriented programming in Perl. They stressed the point that Moose is not a toy or experimental, but very stable and being used in production. There are a few performance penalties to pay, but they are working on a compiler to reduce those.

Yoshinori Takesako: How to defend Apache/CGI against multi-byte XSS attacks

Poorly written or unmaintained web applications in combination with newly discovered scary browser vulnerabilities leave web site operators in a tough spot. Takesako-san introduced some Apache modules that can be installed as a workaround until the problem is being fixed. Kind of like a firewall, but instead of protecting your server from malicious access, it protects the browser from your malicious server.

Tatsuhiko Miyagawa: 20 modules I haven't yet talked about

Entertaining talk as Miyagawa-san goes through the back-catalogue of his extensive oeuvre. He actually cut it down to ten, because of timing constraints.

Lunch break

Good old university cafeteria food. Ample, tasty, and cheap. Taken open-air in the sun.

Ingy döt Net: Perl Love for Javascript Hackers

Ingy shows how to use Makefiles to make web pages, and how the much-maligned Template Toolkit can be used in JavaScript, too, using Jemplate.

Jesse Vincent: Everything but the secret sauce

Best Practical are open-sourcing all their tools (keeping only the secret sauce they use to hook them together), and Jesse showcased a number of them, including their Perl application packaging system Shipwright.

Faiz Kazi: From POE to Erlang

Faiz proposes that even though Perl now (ever since 5.8) has its own support for threads (which have not been met with much in the way of adoption), the best way to do concurrency in Perl remains the venerable and under-appreciated Perl Object Environment, which is based on actors passing messages (as opposed to threads sharing memory). Erlang is a language built upon that principle, and is said to scale phenomenally for concurrent applications, which may become more important as CPU speeds have stopped increasing as dramatically as they used to (to compensate, computers have more cores now).

For the third year in a row, the Shibuya Perl Mongers brought YAPC::Asia to Tokyo, this time on campus at the Tokyo Institute of Technology, which I hereby proclaim to be the best venue yet. Today's Day One was the second day actually, as there were also some additional talks yesterday (including one by co-worker-on-sabbatical Faiz), but I was tied up at work and could not go. Equally unfortunately, there is no T-shirt this year, and I could not get WiFi to work on the XO-1.

Larry Wall: A Standard That Is Meant To Be Broken.

A keynote that was a bit more on the technical side (they are usually more political or philosophical), in which $Larry talked about internals of the Perl6 parser he has been working over the last year. He stressed that there are not really any built-ins, and that the parser can be extended or otherwise modified at runtime to support all kinds of languages. And there is still a lot of dwimmery going on, such as the longest token matcher, which provides a sensible default behaviour when trying to parse a complex grammar.

Kang-min Liu: Continuous Testing.

A presentation of Test::Continuous, which polls your source directory for changed files, so that relevant tests can be run when you update them.

Lunch break

Another accidental lunch with Larry Wall (and his wife, and Faiz), at Matsuya.

Jose Castro: Perl Black Magic.

A very entertaining introduction to obfuscation and Perl Golf.

Ingy döt Net: JavaScript Love for Perl Hackers.

Ingy has ported the very popular jQuery to Perl, so that you can do web scraping with this powerful DOM query language. He also introduced his latest slideshow system Vroom, and proposed to use CPAN to distribute JavaScript libraries as well.

Leon Brocard: Working in the cloud.

Leon explained how his company is operating without any hardware himself, having outsourced everything to Amazon, Google and the like.

Jesse Vincent: Step 3 - Prophet.

In what was the most exciting talk for me, Jesse talked about Best Practical's experimental peer-to-peer replicated database. This fits in very nicely both with my recent musings about the future of databases, and my recent migration from Subversion to the distributed Mercurial source version control system, which works very similarly. The biggest challenge of these peer-to-peer systems is how to resolve conflicting updates.

Chia-liang Kao: Running A Perlish Small Business.

CL in his spare time runs AIINK, a small printing company, and explained how he quickly cobbled together the infrastructure for it with Perl.

Makoto Kuwata: The Fastest Template Engine in Perl World.

Yet another templating engine, but these things get incrementally better, and this one is pure Perl, very fast, feature-rich, and has an extremely clever (if you know Japanese) name: Tenjin - Template Engine.

Lightning Talks

More templating engines, YAPC financials, RSS readers, Perl6 signatures in sane Perl5, Japanese CPAN authors, from POE to Erlang (prologue), an OS written in Perl, web services, and crazy HTML hacks

Starting with Perl 5.9.3, Perl finally has an official switch statement (in versions before there have been various hacks to emulate it, none of them pretty). The long wait has paid off, the Perl version is packed with dwimmery ("do what I mean").

/* C switch statement */
switch (data) {
	case 1:  printf ("one\n"); 
	         break;
	case 2:  printf ("two\n"); 
	         break;
	default: printf ("something else\n");
	         break;
}

In order to use post-5.8 features you have to enable them with the new feature pragma. This assures that the new keywords do not conflict with existing code. The keywords are not called switch and case, but given and when. A straightforward translation of above C code (also using the new say function, which is Perl's println) is the following:

use feature qw( switch say);

given($data) {
	when (1) { say 'one' ; break; };
	when (2) { say 'two' ; break; };
	default  { say 'something else'; break; };
}

The first improvement that has been made is to make the break optional. Whereas C and Java will fall through to the next option, which is usually not what you want, Perl will break out by default.

given($data) {
	when (1) { say 'one' ; };
	when (2) { say 'two' ; };
	default  { say 'something else';  };
}

Conversely, you cannot get the C behaviour of falling through to the next option. There is a continue keyword, which will go to the next when, but that condition will still be checked.

# this code does not work !
# it will say "something else"
given($data) {
	when (1) { continue; };
	when (2) { say 'one or two' ; };
	default  { say 'something else';  };
}

But the when clause is quite smart and can do more than simple literal comparisons, letting you say

# this works
given($data) {
	when ([ 1, 2]) { say 'one or two' ;  };
	default  { say 'something else';  };
}

This smart matching feature supports many different types of comparisons and it can also be used outside of when using the new ~~ operator.

Finally, you can combine the switch with a loop:

my $count = 0;
for (@array) {
   when ("foo") { ++$count }
}
say "\@array contains $count copies of 'foo'";

Ingy döt Net: Kwiki and the Symlink

Proposing a new way to package and install Perl modules with their dependencies, using Subversion and symbolic links. Does not work on Windows, but is much easier (and cleaner) than a normal CPAN install. Inspired me to download Ingy's slideshow system Spork to make the slides for my lightning talk.

Emerson Mills: Virtualization and Package Deployment with EC2

Introduction and demo (that did not work because the Internet connection was down) of the tools that come with the Amazon EC2 hosting service

Mark Jason Dominus: Higher-Order Parsing techniques for Perl

Demonstration how to build a parser from scratch using only functions and function-generating functions, based on MJD's acclaimed book

Dan Kogai: perl I18N in 20 minutes

Showcasing the Unicode capabilities in Perl and comparison with other languages

Naoya Ito: Network programming with Perl

Everything you ever wanted to know about BSD sockets, epoll, kqueue and the like.

Ben Trott: Everything Vox

CTO of Six Apart shares some tricks how to make their sites scalable and fast: Caching, partioning, batch processing. Apparently these big web sites do not really use RDBMS anymore, at least not as an abstraction layer for the physical data storage, precisely because sufficient performance cannot be achieved without complete control over physical storage and access paths.

Tomohiro Ikebe: Inside livedoor 2006-2007

A similar talk to the previous one, about the architecture that livedoor (they insist on the casing) is using, and the changes there over the past year (strictly confined to technology).

Marty Pauley: Perl Worst Practices

Java was designed for stupid people, Perl for clever people. Fortunately for Java, it can (and is) also be used by clever people. Unfortunately for Perl, it can (and is) also be used by stupid people, which is dangerous.

Audrey Tang: Perl 6 Today

An overview/roadmap about the state of Perl6 development. Nothing too specific, and nothing really new, either. We'll just have to wait until Christmas.

Jesse Vincent: Abusing Domain Specific Languages for Fun and Profit

Three examples of domain-specific languages that are implemented in Perl and can be parsed using the Perl compiler, even though they do not look like Perl at all anymore.

Kang-min Liu: Asynapse - The missing links between servers and clients

An API to easily expose server-side code to Javascript applications (or other REST clients). Also, a converter for Mac Interface Builder files to YUI or XUL.

Dave Rolsky: Patterns in Perl

Design patterns from the GOF book, and why some of them do not really make sense in Perl, as they seem to only exist to work around limitations in languages like C++.

Yoshinori Takesako: s/ Perl5 Regular Expression / Perl6 Regex and Rule /mixes;

An overview of the new regular expression features in Perl6.

Daisuke Maki: (Ab)?using Class::C3

An introduction to a popular library for method dispatch when using multiple inheritance.

Chia-liang Kao: Pushmi - Subversion replication system

A subversion mirroring system for people that do not want to use SVK but still need to have at least read-only access when your Internet is down due to an earthquake.

Gosuke Miyashita: Assurer - a pluggable server testing/monitoring framework

Introduction to Assurer, a server health monitoring system

Lightning Talks

Twelve five minute talks, including three extremely funny ones, and mine, which was not nearly finished when time was up. Three parts: exposition, explanation, live demo. Gong after the first act. Should have practiced it beforehand.

Shibuya.pm really needs to find bigger venues for their events. Tech Talk 7 is next week and it took about an hour from the announcement of the registration page to all seats being gone, which means that I, who unfortunately gets to check my email only about once a day now, could not get my name in. I am going anyway, and hope they still have standing space.

- Do you have any experience (funny or not so) that you had while making
"PerlPad" and that want to share with us ??

The most amazing thing that ever happened was that an Argentinian Perl
Monger wanted to interview me about it. I am still flattered ;-)

Esta vez es la hora para la plataforma Apple y Mac OS X, así que es tiempo para PerlPad (http://perl-pad.sourceforge.net/), que ofrece ejecución de código Perl como un servicio de Mac OS X (system service). De esta forma se puede ejecutar código Perl desde casi cualquier aplicación Mac OS X.

Pero dejemos a su creador, Thilo Planz, contarnos su historia. [HTML] [PDF]

Today there were two tracks of sessions and I attended as much as I could. I guess I got enough inspiration for a few more episodes of the thousand and one reasons to Love Perl. So many cool things, so little time...

• Atsushi Kato: Glue: Separate Application Logic from Code
• Dan Kogai Perl 5.8 and Unicode: Myths, Facts and Changes
• Damian Conway: Perl 6 Update
• Dave Rolsky: Perl DateTime project
• Audrey Tang: Learning Haskell
• Chia-liang Kao: SVK: Version Control without the Headaches
• Marty Pauley: 混合語 (Kongougo)
• Damian Conway: Sufficiently Advanced Technologies
• Ingy döt Net: Wikiwyg
• Lightning Talks
  • Audrey Tang: Visual Basic Rocks
  • Dave Rolsky: What is a Data Model? The Relational Model; SQL is not Relational
  • Tokuhiro Matsuno: DebugScreen meets PadWalker
  • Shinya Hayakawa: 続my変数マニアックス(仮)
  • Masayoshi Sekimura: できる Perl debugger
  • Daisuke Murase: CPAN Recent Changes
  • Tokuhiro Matsuno: 曜日判定における言語の壁問題
  • Kazuho Oku: rapide - Remote Apache IDE
  • Kenichi Ishigaki: Let's play Sudoku
  • Yoshinori Takesako: ppencode for Perl6
  • Ingy döt Net: Template Toolkit + Javascript = Jemplate
• Larry Wall: Keynote

Trying to blog a conference without a laptop is difficult/futile, so here are just a bunch of links to the session schedule (there being only one track today, I did not have to choose between parallel sessions, and could attend all of them).

• Audrey Tang: Introduction to Pugs
• Ingy döt Net: Module::Compile - A Shrew(e)d Axe
• Leon Brocard: mightyv
• Yukihiro Matsumoto: Ruby on Perl(s)
• Lunch with @Larry, who just happened to walk into the Korean restaurant where I was sitting.
• Tatsuhiko Miyagawa: Plagger - RSS/Atom remixing platform
• Batara Kesuma: mixi.jp: ChangeLog
• Dave Rolsky: JSAN and AJAX
• Jesse Vincent: ♨ Jifty
• Masayoshi Takahashi: the first postmodern computer language vs the first Babel-17 computer Language: Ruby
• Damian Conway: Perl Best Practices

A rather unique feature of Perl is Taint Mode. If the Perl interpreter is running with Taint Mode enabled, it treats all user input as insecure and refuses to do certain dangerous operations with it. All command line arguments, environment variables, locale information, file input, and the results of certain system calls are considered user input, any command that invokes another process, modifies files, directories or processes is considered dangerous. In addition to that, environment variables and the current working directory are not referred to when loading modules, and directories that are not absolute or writeable by other users are disallowed in the PATH environment variable. Any attempt to use tainted data in the situations outlined above results in immediate termination of the Perl program. The only way to get around this is to untaint the data, which can be done by matching it against regular expressions describing the acceptable values.

Note that Taint Mode can only encourage programmers to include validation of user input in order to prevent security problems: It forces the programmer to write some validation code, but to make those checks effective remains the developer's responsibility.

Also note that Taint Mode is a mechanism to protect your program from causing damage when given unexpected input. It is not a mechanism that makes it safe to run untrusted programs. That problem is addressed by the Safe module, Perl's way of sandboxing code.

I am the maintainer for a Perl database abstraction module, that automatically creates the necessary SQL to access stored procedures. Testing code that interacts with a database can be tricky. You need to have instances of all the supported database products available. You need to set up the connection passwords for the test suite. Certain tests may depend on the database being in a certain state, so you need to put it in that state. Basically you want to have complete control over the database for the purpose of running your tests without risking to damage any important data that may be stored in there. And of course, the tests should still be able to run automated, for example as part of the CPAN install process.

Enter DBD::Mock. It is a fake database driver that just accepts and records all the commands you throw at it. You can also prep it with mock data that it should return. After running a transaction using DBD::Mock (instead of the regular database driver), you can check the logs it collects and verify that the SQL your code has issued came out as expected. Obviously, you cannot check how a real database would have reacted to it, but testing the database is not your job anyway: You should only really be concerned that your SQL is correct (which you have to verify in some other way). There are certainly cases where you need to do the whole real-world round-trip, but for an SQL-generating module such as mine, DBD::Mock already takes you a long way.

I attended Shibuya.pm's Tech Talk #6 yesterday. As far as I can tell the audience was made up by the same crowd that went to Blog Hackers Conference earlier this year, and the Takahashi method remains popular. It was also a good chance to meet up with former co-workers from both Gaiax and Dreamarts.

The six talks were about life at Six Apart (company culture, development process, the software that they make and the software that they use), Catalyst (a Perl web framework that looks like a cross of Struts and Rails and was the topic of two talks), prototype.js (my favourite function therein received mention), Perl at the ICFP programming contest, and an introduction to Perl6/Parrot/Ponie/Pugs.

Every year, Larry Wall, creator of Perl, delivers his State of the Onion Address, in which he talks about the general state of the Perl community. This time he likens the Perl Cabal to a spy agency as seen in James Bonds movies. Very entertaining.

The official development environment for Apple's Mac OS X is called Cocoa. Cocoa, which has its roots in NeXTSTEP, is a collection of libraries that are used to create GUI applications. The native language for Cocoa is Objective-C, and since this is a language no one outside the NeXT community knew at the time Mac OS X was released, Apple is also offering a Java programming interface to it.

CamelBones is an open-source project that lets you access Objective-C libraries from Perl, and Perl code from Objective-C, allowing you to write native Mac applications in Perl. There are a lot of tricky technical details involved to make this work, but the creator of CamelBones has them all covered and you can basically access all the functionality available to Objective-C from Perl. For end-users, the applications you create are not different from "normal" Cocoa applications. Unlike in older versions, current CamelBones does not even have to be installed on the user's machine anymore: He can just run the application out of the box.

Objects and methods are all automatically bridged both ways, so that you can just refer to the official Cocoa documentation and use exactly the same patterns outlined therein. This is especially impressive (and a testament to Perl's excellent ability to act as glue code to native libraries) when you compare this to Java: Whereas every Objective-C framework becomes automatically accessible to Perl (without having to be registered with CamelBones in advance), the Java interface requires hand-crafted wrapper code, so that Cocoa-Java (an officially supported environment!) always lags a few months behind where new functionality is concerned.

Furthermore, you get to use Apple's official development tools (XCode), which are really quite nice. You even build your user interfaces with Interface Builder, same as you would do in Objective-C.

CamelBones, of course, is not a cross-platform environment. It pretty much ties you to Cocoa and thus the Mac. On the other hand, it apparently also works with GNUstep, so that it might actually get useful on other systems as well.

PerlPad, of course, is built with CamelBones.

My involvement in the Perl community has so far been a purely virtual one (via CPAN and Perlmonks), but I would really like to visit a conference or join a Perl users group. I made a first step into this direction by visiting the Blog Hackers Conference 2005 today, a three-hour event with speeches centred around O'Reilly Japan's Blog Hacks book. There were two longer presentation by the book's two authors and a number of lightning talks, held by people working for companies like Hatena or Livedoor. Everything was in Japanese, of course (the rate of foreigners was at most three in about 150, same as the rate of women), so my understanding was limited to the technical aspects and all the jokes eluded me (there was a lot of laughter, so I guess it was funny).

The conference was organised by O'Reilly, the IT school were it took place (Digital Hollywood, Graduate School of Digital Content) and the Shibuya Perl Mongers, who I am slowly trying to approach, which is difficult, since they do not seem to have regular meetings, and if they do, they announce them on Mixi (an immensely popular example of these silly social network services where you cannot read anything unless you sign up, a concept that I protest by not signing up) rather than on their web-site or mailing list. My mid-term goal here is the YAPC::Asia, which is rumoured to be hosted by Shibuya.pm next year.

Ever since its inception just a few weeks ago, the new Pugs project has captured the minds of the Perl community. Among those of feeble faith Perl6 has taken on a distinct vaporware image, being under development for five years now without even an alpha release yet (the efforts so far have been mainly focused on creating the language specifications in a series of Apocalypses, Exegeses and Synopses, as well as on getting the underlying runtime engine right). It was not so helpful for the discussion that you could not try out the shiny new features of the Perl 6 language (which significantly differs from Perl 5) at all. But this has changed now, thanks to Pugs.

Pugs is a project of wonderful eccentricity. It has been single-handedly created by a lone hacker (apart from the official Perl6 project), has (or at least had) a funny name (Perl6 User's Golfing System), is implemented in Haskell, initially limited itself to a functional, side-effect-free subset of Perl6 called Featherweight Perl, has a version numbering scheme that in the Knuthian tradition converges against 2π, its own series of documentation called Apocrypha and an ugly dog as its logo.

Pugs provides a Perl6 interpreter that already implements an impressive (given the youth of the project) part of the language, definitely enough to start playing around with it. Some people expressed concern over whether Pugs will not draw away resources from the official effort, but I should think the opposite is the case. Pugs provides an easy hands-on entry for those who want to start working with Perl6, and it can be used to create test cases for the real compiler (which of course will be written in Perl6, as any good compiler can be expected to compile itself).

And now, for your amusement, the Pugs roadmap, and a working Perl6 code example (as you can see the new syntax is scary at first):

    * 6.0: Initial release.
    * 6.2: Basic IO and control flow elements; mutable variables; assignment.
    * 6.28: Classes and traits.
    * 6.283: Rules and Grammars.
    * 6.2831: Role composition and other runtime features.
    * 6.28318: Macros.
    * 6.283185: Port Pugs to Perl 6, if needed.

#!perl6
use v6;

multi sub quicksort ( ) { () }

multi sub quicksort ( *$x, *@xs ) {
    my @pre  = @xs.grep:{ $_ < $x };
    my @post = @xs.grep:{ $_ >= $x };
    (@pre.quicksort, $x, @post.quicksort);
}

(1, 5, 2, 4, 3).quicksort.say;

Unlike for example Java or .NET, which have some big industrial players promoting them, Perl is mostly community-driven. This community is quite active, and also includes a number of very bright people, who at the same time are also very helpful and willing to share. One manifestation of this fortunate situation is the previously mentioned CPAN public code library, another one is the Perl Monks community web site. The Perl Monks are a great example of a community site that works. Everyone can ask questions, and usually within an hour receives high quality answers: Newbies are (unless they seem too lazy or rude) given patient explanations, the more tricky questions have good chances to be answered by true experts of the field. There is also an integrated chat room for even faster feedback. But the Monastery is not just a Q&A site (although it serves that function extremely well and is not even plastered with ads like many other sites), there are also sections for discussions about Perl, for poetry, for news, for publishing code snippets, book and module reviews, even for obfuscated code (a special tradition that makes fun of the fact that Perl does not force the programmer to code in a readily readable way). And then of course, there is the whole Monastery theme which adds a nice touch.

Sometimes your Perl script is just that: a small script for a specific task that is contained in a single file. Because a single file is easier to deploy and track than applications that span many files (program, configuration and so on) you want to keep it small. However, at the same time, you want to have the script read something from a data file, for example a template for the report it produces, or a list of people to email the report to. Using Perl's many quoting functions (especially for multi-line strings) you could just throw that data into a variable. But sometimes a file handle is more convenient, if only because you can easily loop over it line by line. No problem, you can just include that data in the same file right after the script and the special __DATA__ keyword. You can read from this __DATA__ section just like you would read from a file:

printf '%03d> %s', $., $_ while (<DATA>)  # special filehandle DATA
__DATA__
Once upon a midnight dreary, while I pondered weak and weary,
 Over many a quaint and curious volume of forgotten lore,
 While I nodded, nearly napping, suddenly there came a tapping,
 As of some one gently rapping, rapping at my chamber door.
 `'Tis some visitor,' I muttered, `tapping at my chamber door -
 Only this, and nothing more.'

Every year, Mark Fowler of the London Perl Mongers puts together his Perl Advent Calendar which introduces a different Perl module every day until Christmas.

Perl has a mechanism that lets you intercept calls to undefined subroutines. You just have to write a subroutine called AUTOLOAD, which will be called instead of the missing subroutine. This feature can be used in various ways. You can for example create a customised error message. Or you can implement subroutines that are loaded on demand: the first time they are called AUTOLOAD creates and installs the subroutine, so that subsequent calls will work the same way as if the subroutine had existed all along (this is probably how this mechanism got its name). I started using it to call PL/SQL stored procedures from Perl.

I have a lot of stored procedures that all look like

function do_something ( some_parameter varchar2, another_parameter integer)
    return integer;

I can call this procedure using DBI

my $sql = $conn->prepare(q{
    begin
        :r := do_something(:some_parameter, :another_parameter);
    end;
});
my $result;
$sql->bind_param_inout(':r', \$result, 10);
$sql->bind_param(':some_parameter', $some);
$sql->bind_param(':some_parameter', $another);
$sql->execute();

It is quite easy to make a generic version of that code, which looks at the function name and the parameters it is given to generate the SQL statement and bind the parameters. Using this as AUTOLOAD in the database access module, I do not have to write any more code (SQL or Perl) to access these stored procedures, and can just do

my $result = Database::do_something(
    some_parameter=>'blah', another_parameter => 123);    

If I add another stored procedure do_something_else, it will automatically become available to the Perl program, which speeds up development quite a bit and makes the application code smaller and more maintainable by reducing redundancies.

my $result = Database::do_something_else(funny_parameter => 12345);    

Regular expressions are the Swiss Army Chain-saw for string manipulations and no one disputes that Perl really shines here. I would say they are almost indispensible when working with text data. Regular expression engines have recently been added to the core Java platform and the Oracle database. I know that I miss them in my Parrot experiments (Parrot already has a working regex engine, with all the low-level opcodes required for that, but no compiler to target it). While Perl did not invent regular expressions, it has made them popular, and Perl's implementation has become the gold standard of the field (the most popular C library for the task is even called pcre -- Perl Compatible Regular Expressions).

Regular expressions are a (rather cryptic) language unto themselves, and I am not going to describe them here in detail. Have a look at the Quick Start Tutorial or the regular Tutorial.

The funny thing is, now that everyone has copied Perl's syntax for regular expressions, Perl 6 is going to throw it overboard and start anew. See Exegesis 5.