minvoke - forcing portability

toshok | mono, geek | Monday, June 29th, 2009

Portability to Mono is pretty easy when writing applications entirely in C#.  Winforms works well enough, just about anything console-based will work, and as long as you avoid some obviously broken areas of the API (System.Management, I’m looking at you), you’re usually ok.

We’ve done much in the area of migration tools, such as providing the excellent MoMA, which offers you a easy way to both get information about your problem areas (if you’re calling into unimplemented apis, making pinvokes, etc), and inform the mono project (anonymously, if you desire) so that we can use the information to prioritize missing features.

But we can only do so much.  Some win32 hackers remain tied to user32.dll, kernel32.dll, and other platform dlls.  It’s rather a shame that a winforms app loses out on mono portability simply because they’re pinvoking SendMessage or something.  If the developer cares about portability you can sometimes motivate them to correct the problem, but sometimes you’re faced with an old, unmaintained app, or a cantankerous developer, or something provided by a faceless corporation with nobody to complain *to*.

This is where minvoke comes in.  Simply put, it uses the insanely awesome Cecil to rewrite pinvokes into managed method calls.

Take the following trivially broken (on linux/osx) code:

using System;
using System.Runtime.InteropServices;
class MyBrokenExecutable {
    [DllImport("kernel32.dll", SetLastError=true)]
    static extern bool QueryPerformanceCounter(out long lpPerformanceCount);

    public static void Main (string[] args)
    {
        long perfCount;

        if (QueryPerformanceCounter (out perfCount)) {
            Console.WriteLine ("QueryPerformanceCounter returned {0}", perfCount);
        }
    }
}

if you compile and run this on linux or your mac you’ll see:

$ mono BrokenTest.exe 

Unhandled Exception: System.EntryPointNotFoundException: QueryPerformanceCounter
at (wrapper managed-to-native) MyBrokenExecutable:QueryPerformanceCounter (long&)
at MyBrokenExecutable.Main (System.String[] args) [0x00000]

In mono svn, mono-tools/minvoke you’ll see MapAssembly.cs, which contains this:

public class Kernel32 {
...
    [MapDllImport ("kernel32.dll")]
    public static bool QueryPerformanceCounter (out long performanceCount)
    {
        Console.WriteLine ("Kernel32:QueryPerformanceCounter called");
        performanceCount = 0;
        return true;
    }
...
}

Note the MapDllImport attribute.  minvoke uses that to match up the entry point and dllname with pinvokes, and rewrites the call to invoke this static managed method instead.

QueryPerformanceCounter’s implementation is not extremely useful at the moment.  In fact, everything in MapAssembly.cs is just stubbed. But all the same, if you run minvoke:

$ minvoke MapAssembly.dll BrokenTest.exe FixedTest.exe
building list of MapDllImports from map assembly MapAssembly.dll
building list of DllImports in input assembly BrokenTest.exe
retargeting assembly BrokenTest.exe -> FixedTest.exe
retargeting reference to method kernel32.dll/QueryPerformanceCounter

and then run FixedTest.exe, you’ll see:

$ mono FixedTest.exe
Kernel32:QueryPerformanceCounter called
QueryPerformanceCounter returned 0

Keep in mind, this is a proof of concept.  It does no type checking of pinvoke signatures, doesn’t handle all the DllImport attributes, and pretty much nothing has been implemented.

My plan is to split up MapAssembly.cs into MapUser32.cs, MapKernel32.cs, etc, and have them implement things in terms of mono’s assemblies (i.e. the SendMessage implementation will call into our managed System.Windows.Forms’s XplatUI layer) wherever possible, and when it’s not possible to completely implement something in managed code, support both linux and osx setups using pinvokes.

I’d love help on it, as otherwise it’ll only get attention when I’m suffering moonlight burnout or have a spare evening (not very often).  I’m definitely accepting patches :)

To take a look at the code, check out the mono-tools module from mono SVN, or grovel around here. Enjoy

new moonlight bits

toshok | moonlight, mono, geek | Monday, May 18th, 2009

We’ve been slaving under the pretty aggressive preview schedule of branch on friday, cherry-pick important changes over the weekend, release monday for a few weeks now.  I think we’re all having a hard time coming up with really noteworthy changes each time, since the time to get fixes in is so short.  We’re pretty much all living on git branches for big changes, and land them monday or tuesday.  Any later in the week and it’s too close to branch time to get adequate testing in.  Last preview was a good example of this.  No huge earth shattering changes, just incremental improvements across the board.  Trouble is, there might be one small piece of moonlight missing that blocks a large site from working properly (or even parsing.)

SL2 Control Toolkit Demo

This week we hit a pretty big milestone which was getting the silverlight2 toolkit control demo site working. Turns out it was a pretty minor piece of missing functionality: In SL2, Microsoft introduced ObjectKeyFrame’s, which allow arbitrary values (basically anything expressable in XAML, from strings to more complex object trees) to be specified as key frame values for an animation.  This is often used to make objects visible using the Visibility enum as opposed to using Opacity.  But the normal rules apply for string values in xaml.  If you’re animating the background property on keyframes between orange and red, so something like:

<DiscreteObjectKeyFrame … Value=”Orange” />
<DiscreteObjectKeyFrame … Value=”Red” />

We need to actually figure out that you’re animating a property with a Brush type, and do the type conversion from string  to SolidColorBrush.  Turns out we have code to do this already, since it’s identical to what styles do.  Alan factored all this code out and made both animations and styles use the new hotness, and all of sudden the demo came to life.  We need this type conversion code because since the Value=”" specified in the xaml is a string, there’s no way to know at parse time that it should be anything other than a string when the animation is run (i.e. you could be animating the text contents of a TextBox).

toolkit-screenshot-thumb.png

On Names…

The release notes also mention something about names, and that’s important to touch on here too.  Silverlight allows you to name elements, which can be found using the helpful FrameworkElement.FindName method.  If you do your development in visual studio or monodevelop, the class associated with your xaml file automatically has fields of the same name as you specify in the xaml.  These are hooked up in generated code using this FindName method.  Another method, Control.GetTemplateChild, does the same name lookup but constrained to the template applied to a control.

There are docs on MSDN about how name registration and lookups happen, but those docs don’t really tell the whole story (and nor should they.)  There are a rather large number of corner cases, so we now have a test in svn called NameTortureTest.  The tests led to the codifying of a number of rules that silverlight 2 seems to follow.  You can find those rules in moonlight svn here.  Moonlight doesn’t yet pass the entire suite of tests, but the ones it does fail are mostly far out there, and we have one test which crashes Silverlight (I should really post that to the forum..)

On Performance…

We’ve definitely been putting functionality ahead of performance, and except for one case, haven’t done many sweeping performance changes during the course of 2.0.  So don’t be surprised when you load up the control toolkit demo (or any site, for that matter) and we’re a little sluggish.  We’ve actually got a surprisingly good handle on the majority of our first round performance problems.  We’ve taken google’s great performance tools, and along with the insanely cool XDEBUG support Zoltan has added into mono-trunk, we’ve got managed symbols appearing in our profile traces.  Once we get functionality nailed down and test suites passing, we can start taking the perf-bat out for a little fun.  I expect us to start swinging around beta time, maybe a little before.  So stay tuned..

license…

toshok | geek | Thursday, May 7th, 2009

I still toy with the idea of going back to law school just for a single purpose:

So I can end as many licensing discussions as I can like this:

<someone> IANAL, but I’m pretty sure the GPL forbids this, and therefore the license is invalid.

<me> well,  you see,  I am a motherfucking lawyer, and you are an ignorant asshat.

I wonder if this seething animosity toward the wannabe pundits/open source zealots would be enough to sustain me until I passed the bar…

libtool, a tool whose time has come and gone?

toshok | geek | Monday, May 4th, 2009

I know it’s an inflammatory title, but I’d really like to know.  Has anyone’s life been made easier by libtool in the past, oh, 5 years?  I would like a success story, because in my own life, libtool is nothing but pain.

Oh, and I should say that’s pain even when using dolt.

Moonlight 2.0 preview!

toshok | moonlight, mono, geek | Monday, May 4th, 2009

Well, we did it.  We finally released the Moonlight 2.0 preview, available here.

We’ve had innumerable hurdles (in the form of technical, legal, and process hiccups) to pass to get to this point.  But now that we’ve finally ironed out the big issues, previews should start flowing along weekly on our way to the 2.0 release.

I’ve tried a few times to come up with a list of detailed release notes, but there’s way too many things to cover.  Looking at the changelogs from when we pushed 1.0.1 to where we are now is pretty daunting.

But from a high level, here’s where we are (this is mostly lifted and fleshed out a little from the release notes on the wiki preview page.):

  • The biggest single point I think is this - we’re finally comfortable releasing a browser plugin containing the Mono VM.  This is pretty huge, and the runtime guys deserve a lot of credit for making it possible.  This means we’ve invested enough time and effort into fleshing out the infrastructure (CoreCLR, as well as our metadata and IL verifier), and getting it to a point where we’re not totally embarrassed to share our work.  Keep in mind, though, that a full security audit has not happened, and that by visiting Silverlight sites you are downloading code that will execute on your system.  There’s more specific information about security on the Moonlight wiki.
  • DeepZoom functionality is working.  You can see it in action by visiting Hard Rock Cafe’s memorabilia site.
  • The Microsoft Silverlight Controls, since they’re open sourced, are bundled with the release so all the controls you expect to see (Button, Scrollbar, etc.) look and behave exactly the same as they would in Silverlight.  This requires a lot of other infrastructure.  Databinding, templates, styles, animations..  everything has to work or nothing works.
  • Negotiated layout.  You no longer have to absolutely position elements inside a canvas.  The controls source release contains many container controls (StackPanel, Grid) that provide alternative (and convenient) layout behaviors.
  • A couple of new development tools are available (if you build from the source) to help building Silverlight 2.0 compatible .xaps.  These include xamlg, respack, and mxap.  Mxap actually provides support for building both Silverlight 2.0 and desktop xaps.
  • mopen (our local application tool) recognizes both desktop xaps and xaml files.  As it runs completely outside of the browser, it has no System.Web.Browser api and no javascript support.  It does, however, provide full access to the .net 2.0 api, include the local filesystem, sockets, the (since it has no CoreCLR) ability to pinvoke, etc.  Everything you’ve come to expect when writing .net apps on linux, wrapped up in a nice Moonlight shell.

Since we started working on Moonlight 2.0, Microsoft has released a beta of Silverlight 3.0.  The differences between 2.0 and 3.0 are much, much smaller than the differences between 1.0 and 2.0, and we’ve been keeping the 3.0 in mind when completing work on various 2.0 features.  Where it made sense to spend a little extra time to get some extra functionality, we’ve added 3.0 features.

The Silverlight 3.0 features available now in Moonlight 2.0 include:

  • Easing functions for animations, including user-supplied ones.
  • SaveFileDialog, a safe way to allow users to save content from Silverlight applications
  • MultiScaleImage (the heart of Deep zoom) API additions (e.g. the AllowDownloading property).
  • MediaStreamSource now supports PCM audio data, RGBA and YV12 video data.  This along with other extensions makes it very easy to write codecs entirely in managed code, that you can then distribute with your xap.
  • WriteableBitmap is supported.

Given the level of change in the code there are going to be regressions from what worked in 1.0.1.  It’s inevitable.  Our test suites aren’t complete, and we aren’t passing everything in them yet.  This is a preview, after all, but in my opinion it’s a pretty exciting taste of what’s to come.

Miguel has some more preview awesome-sauce over at his blog.

Next Page »

Powered by WordPress | Theme by Roy Tanck