use types;

YAPC::EU 2011

rurban - Reini Urban
vienna.pm => houston.pm

Goal

Provide type semantics to enable more compile-time optimizations, to make Perl and compiled Perl smaller and faster.

Outline

• Status Quo: CORE types

  my TYPE $i; my My::Class $o; # Declaration only
  package TYPE must exist

• Optimizations planned for 5.16 - 5.18

  smaller and faster, ensured at compile-time:
  faster method calls, fix types, declarations via class and attributes, typed arrays + hashes, perfect hashes.

• Artur Bergman's types module

  Declare and check type safety. int, double, number, string. params and return values.

About me

About me

Architect. Mainly LISP for several years. Perl as better build system.

rurban maintains cygwin perl since 5.8.8 and some modules: illguts, B::* => 5.10, mainly the "compiler".

No perl dayjob, just for fun. Until now.

In the future a lot more time for the compiler and CORE. Left AVL (automotive industry), went to cPanel to improve the compiler.

Perl's Type System

my int $var; # or our

# my :attribs are not known at compile-time, only our.
$ perl -cw -e 'package int; package main; our int $i:const;'
    Invalid SCALAR attribute: const at -e line 1
$ perl -cw -e 'package int; package main; my int $i:const;'
    -e syntax OK

FETCH_SCALAR_ATTRIBUTES => New CHECK_SCALAR_ATTRIBUTES callback for my.

Perl's Type System

Yes, the language already has one. CORE:

  my TYPE $var;
  sub name (proto) {} => proto: &name->pv

Only a few out-of-CORE implementations:

Moose, fields, types, typesafety, Lexical::Types, Attribute::Types (run-time)

Params: Devel::Declare, Params::Validate, Params::Classify
Objects: Class::Meta, Moose

Prototypes

No standard prototype language, no CPAN prototype parser!

# types 0.x
sub foo (int $foo) { my ($foo) = @_ } # return type implicit. 
#No optree injection as in Devel::Declare

# typesafety
sub foo (int; int, undef) # returns int

# types 1.0
sub foo (int $i => int) { return 2 * $i } # return type explicit

Prototypes

No standard prototype language, no CPAN prototype parser!

# Devel::Declare, Method::Signatures
method hello (Str :$who, Int :$age where { $_ > 0 }) {
  $self->say("Hello ${who}, I am ${age} years old!");
}

# Moose
has 'accessor' => (isa => Int, is => 'rw', default => 1);

Perl's Type System

At first a look at CORE types, not the language:

SCALAR (non-strict, VB-like): IV, NV, UV, PV, ...

ARRAY (non-strict): AV

HASH (non-strict): HV

Objects ("stashes", @ISA, declaration)

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

multiple types (IV,NV,PV) per variable, context dependent.

internally upgraded: IV => PVIV => PVNV

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

multiple types (IV,NV,PV) per variable, context dependent.

IV: integer

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

multiple types (IV,NV,PV) per variable, context dependent.

Typed IV: integer unblessed, untied. my int $i; ~SV_SMAGIC => SVf_FAKE: no head

=>

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

multiple types (IV,NV,PV) per variable, context dependent.

internally upgraded: IV => NV

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

multiple types (IV,NV,PV) per variable, context dependent.

internally upgraded: IV => NV => PVNV

Perl's Type System - SCALAR

SCALAR (non-strict, VB-like)

internally upgraded: IV => PVIV => PVNV => "Objects" => Tie

Perl's Type System - SCALAR

"STASH" - Hierarchical symbol table,
used as package name for Objects. i.e. "Class pointer"

Perl's Type System - ARRAY

ARRAY (non-strict)

Keys are integers, Values any SCALAR

Perl's Type System - ARRAY

ARRAY (non-strict)

Keys are integers, Values any SCALAR

@a = ("any", 1, 2.5006, \&code, ['arrayref'], {hashref=>1});

Perl's Type System - ARRAY

ARRAY (non-strict)

Keys are integers, Values any SCALAR - flexible, untyped, big

@a = ("any", 1, 2.5006, \&code, ['arrayref'], {hashref=>1});

Typed ARRAY

Typed ARRAYs (optional, faster, less space)

Keys are integers, Values of one type only (native)

  my int @a; $#a=256;     # pre-allocate: 256 * sizeof(IV)
  my int @a = (1);           # non-fixed size: 3 * sizeof(IV)
  my int @a :const = (1); # fixed size: 1 * sizeof(IV)

Untyped:

  my @a; $#a=256;         # pre-allocate: 256 * sizeof(SV)

Typed ARRAY

  my int @a;

AvTYPED && (IOK | NOK | POK)

Typed ARRAY

  my int @a;

Declaration already possible now!

  perl -we'package int; package main; my int @a;'
  -e syntax ok

int must be a package definition.

Note: @a is currently declared as int, but not blessed.

Perl's Type System - HASH

HASH untyped: flexible, but big

Keys any SCALAR, Values any SCALAR

  %h = ('any' => ["any", 1, 2.5006];
        1     => 1,
        1.005 => 2.5);

Internally: Keys are stringified.

Perl's Type System - HASH

HASH untyped: flexible
Keys any SCALAR, Values any SCALAR

Typed HASH

HASH typed: fast, small

Keys STRING only, Values typed

  my string %h; $h{'x'} = 'string';
  my int %h;    $h{'x'} = 1;
  my double %h; $h{'x'} = 1.0;

Typed HASH

HASH typed: directer access, small
Keys STRING only, Values typed

Typed HASH

HASH typed: fast, small

Keys STRING only, Values are typed.

  my int %h; $h{'any'} = 1;
  
  $h{2} = 1;
    => Wrong type: hash key no string
  
  $h{'2'} = '1';
    => Wrong type: hash value no int 

Perfect HASH

Perfect hashes - guaranteed O(1) lookup, dynamic hash function generation, no collisions
Library cmph BDZ algorithm ("RAM hashing")

• my %h :const; => untyped perfect hash of unknown size
  
• my %h :perfect; => writable perfect hash
• study %h; => optimize lookup function to perfect hash.

study untyped hash => copies the old perl hash (HV) to new perfect hash (PH).

Perfect HASH

Perfect hash. Should be typed to optimize implementation.

  my int %h :const = ('my' => 1, 'your' => 2);

:const hashes are always :perfect
No need to study with :const and init on declaration.

  my %h :const = ('my' => 1, 'your' => 2);
  # untyped: value = SV

Perfect HASH Idioms

- Avoid copy from perl hash to perfect hash.

  my int %h :perfect;
  $h{$_}=1 for @h;
  study %h;

Perfect HASH Idioms

- Declare size in advance.

  my int %h :perfect;
  length %h = length @h;         # not legal yet, but should be
  $h{$_}=1 for @h;
  study %h;

Perfect HASH Idioms

- :const hash with computed key => values, without study

Idea 1

  my int %h :const;
  length %h = length @keys;       # not legal yet, but should be
  $h{$_} = $i++ for @keys;

Init until length is filled

Perfect HASH Idioms

- :const hash with computed key => values, without study

Idea 2

  my int %h :const = map { $h{$_} => $i++}  @keys;
=>
  my int %h :const;
  { $h{$_} = $i++ for @keys; }

Initialization on next expression, usually a block.

Perl's Type System - OBJECTS

OBJECTS: typed, but dynamic

run-time changable, mostly no compile-time optimizations possible.

Features: STASH ("class hash"), @ISA (mro), DESTROY

Perl's Type System - OBJECTS

OBJECTS: typed, but dynamic.

Class by STASH, Inheritance by @ISA (mro), magic DESTROY and CLONE methods.

Four method calls possible:

• Class->method()
• $object->method()
• Class->$method()
• $object->$method()

Compilation of a function

Class::sub(ARGS...)

pushmark
ARGS...
gv => GV *Class::sub
entersub

Class->method()
$object->method()
Class->$method()
$object->$method()

Compilation of a static method call

Class::sub()

Class->method(ARGS...)

pushmark
const => PV "Class"
ARGS...
method_named => PV "method"
entersub

$object->method()
Class->$method()
$object->$method()

Compilation of a method call

$object->method(ARGS...)

pushmark
padsv => GV *object
ARGS...
method_named => PV "method"
entersub

Compilation of a method call

Class->$method(ARGS...)

pushmark
const => PV "Class"
ARGS...
method => GV *method
entersub

Compilation of a method call

$object->$method(ARGS...)

pushmark
padsv => GV *object
ARGS...
method => GV *method
entersub

Optimization of a static method call

Class->method()

pushmark
const => PV "Class"
ARGS...
method_named => PV "method"
entersub
=>
pushmark
const => PV "Class"
ARGS...
gv => GV *Class::method
entersub

Optimization of a static method call

Class->method(...) => &Class::method('Class', ...)

if defined &Class::method

or package Class :locked or in Moose immutable.
i.e. not changed at run-time.

=> 4% faster method calls.

Note: @Class::ISA :const = qw(bla); does not help.

Optimization of other method calls

Dynamic method calls are possible to optimize in a similar way, if the object is declared - known class at compile-time.

  my My::Class $object = My::Class->new;
  $object->call();

  my My::Class $object = My::ParentClass->new;
  bless $object, 'My::Class';
  $object->call();

Inherited methods are optimizable if all classes in path to the final class are :locked, resp. immutable.

Summary so far

All this is possible now, without changing the language.

Just optimized implementations are missing.

I heard that in July 2011 Moose methods of immutable classes are going to be inlined, but what I saw so far it's not using optree changes like these to speed it up.

More optimizations

More compile-time optimizations.

:const for variables

:locked for packages: const @ISA, no run-time created methods.

use types;

types is Artur Bergman's compile-time checking attempt from 2002, after the compiler, B::Generate and optimize.

And before optimizer, which uses types to improve the optree.

use types;

types does compile-time type-checking only.

compile-time type-optimizations in optimizer.

Problem: slower, not faster.

use types;

The idea is to make programs with use types;

faster, not slower.

And define basic scalar types from CORE

int, double and string

B::CC

The same types and declarations are used in B::CC also to optimize types even further.

B::CC - optimizing perl compiler

B::CC also needs a syntax to optionally declare simple types:

int and double (strict)

So far it was done by magic variable name suffices: $a_i, $n_d;

faster, much faster

B::CC - User Type declarations

Strict types as class, and optional type hints as attributes.

  my int $foo :const;

With use types you get type-declarations,
partial type-safety, and
type optimizations at compile-time.

SUMMARY

    my double $foo = "string"; #compile-time error 
    sub foo (int $foo) { my ($foo) = @_ }; 
    foo("hi"); #compile-time Type mismatch error 
 
    my int $int; 
    sub foo { my double $foo; return $foo } 
    $int = $foo; # compile-time Type mismatch error 

    my int @array = (0..10); # optimized internal representation 
    $array[2] = 2;       # no SV, just the raw int at slot 2. 

SUMMARY

    my int @array :const = (0..10); 
      # even more optimized internal representation 
    $array[2] = 2; # int @array is readonly 
    $array[2] = '2'; # compile-time const error 

    my string %stash :perfect =  
      (foo => 'str1', bar => 'str2'); # perfect hash (stashes)
    ; # more init... 
    study %stash; # initialization finished 

    print $stash{foo}; # faster lookup O(1) 
    $stash{new} = 1;   # compile-time Type mismatch error

SUMMARY

    my int $i :double;  # declares a IV with SVf_NOK.  
    my $i:int:double;   # same but without type-check 
    my int $i;          # declares an IV.  
    my $i:int;          # same but without type-check 

    my int $i :string;  # declares a PVIV. 
    my int @array :unsigned = (0..4);  
        # Will be used as c var in faster arithmetic and cmp. 
        # Will use no SV value slot, just the direct value. 
 
    my string %hash :const = (foo => 'any', bar => 'any');  
        # declare string keys only 
        # generate as read-only perfect hash. 

Questions?

Questions?

http://blogs.perl.org/users/rurban/2011/02/use-types.html