Lycangeek

Visual Programming

There's an interesting little discussion over on slashdot regarding visual programming. The original poster mentioned that while there were some visual languages - mentioning LabVIEW - none were considered mainstream. The discussion that followed I think really missed the point (but maybe that is just me).

Whether we like it or not, programming as it is done today is just way too hard to get to the next level. By this I'm not talking about mainstream applications (think MS Office, Orbitz website, etc), but massively parallel systems to simulate biological systems, the weather, etc. Even mundane applications need a lift to the next level in order to handle concepts such as self-healing (no, Clippy didn't help).

Consider the following:

1. Multi-threaded programming is hard, even for experts. Trying to keep in mind ABA, memory barriers, reentrancy, etc. while getting your main job done is not a simple task.

2. Performance is hard. As Rico Mariani says, you must measure, measure, measure. It is almost impossible to predict where your bottlenecks are. But consider this example of performance problems - cache-misses in the processor cache. If the next instruction or piece of data isn't already loaded into the processor, you are going to be sitting around for THOUSANDS of cycles waiting for it to come in. So - how to avoid that? Keep data and code together that reference each other. How to do that in a large application? There's not a simple answer. And besides - YOU SHOULDN'T HAVE TO CARE!

3. Errors are hard. Of course it is easy to generate errors - most programmers do it without thinking :) but what about handling errors? Pick some random piece of code in your application. Now - what would happen if your disk suddenly ran out of space? What about if the network disappeared? Got an out of memory error? How would you respond? Should this piece of code handle it or should you let it flow up to the caller? Now - realize that these types of errors could occur just about anywhere in your code. Have you thought through all possible conditions?

So what is the answer? If I knew that I would be writing this from the Caribbean rather than Austin. But I do know it will involve constantly reinventing the way we program.

Is LabVIEW the answer? In the context of this discussion, of course not. No language that exists today really addresses all the issues I mentioned. But I do believe that humans are visual creatures and that would seem to point to visual languages as being the right research direction.

Of course, I think that LabVIEW (G) is a great language. It eliminates a lot of complexity that text languages have, such as multi-threaded programming. All you have to do is look at the FPGA solution to see how LabVIEW can let a non-expert create powerful applications.

So, my question is what do you think? Where do you think LabVIEW provides the best solution? Where is it just too hard to express an idea?

Fusion Log Viewer

In previous posts I mentioned the Fusion log viewer (FUSLOGVW.EXE). Now it’s time to talk about how to use it.

It’s important to realize that there are already a lot of sites that talk about this tool, so this isn’t the final word on it, but I’ll focus on the parts that I’ve found useful when debugging assembly load problems. If you’re interested in other sites, here are some good ones: MSDN Article on the tool itself and Junfeng Zhang’s blog about fusion logs.

Using the Fusion Log Viewer

The first thing is to get the logger to record all log attempts – not just failures. The reason is that in certain cases (such as when the assembly is found via an assembly resolver) Fusion first fails to find the assembly, logs the fact, and then tries another track and succeeds. Thus, just because you see a failure doesn’t mean there was one.

Standard Warning – This involves editing the registry. Please be extra extra extra extra…big breath…extra extra extra careful when editing the registry. Do the right things wrong and you will be reinstalling your operating system.

First fire up regedit.exe and go to HKLM\Software\Microsoft\Fusion. If ForceLog isn’t in that key, create it as a DWORD. Either way, it should be set to 1. The next value you can also set via the GUI but you are here anyway, so go ahead and set the LogFailures to 1. Close regedit.

Warning: You are now going to be generating a lot of log entries. You might want to turn on ForceLog only when debugging a problem.

Now fire up the logger (FUSLOGVW.EXE) and click Delete All. This is mainly to start with a clean slate. Unfortunately the logger is a bit primitive and so it can be difficult to follow the log messages (for example, you can’t sort by date/time of the entry). Best to start with a clear slate.

Now go back to your application (I’m assuming that if you are reading this blog that the app would be LabVIEW calling .NET) and run it. Switch back to the viewer and hit refresh. For the sake of this discussion, I am running LabVIEW 7.1 under Windows XP and running the calculator example.

You’ll now see several entries of all assembly loads that took place. Again, the viewer is a bit primitive so it’s hard to tell what worked and what didn’t. But you can see some important information. First, notice that for a LabVIEW application, there are different “Application” names that are all LabVIEW. The obvious one, LabVIEW.exe, represents the DefaultDomain AppDomain and is where we do all of our edit mode reflection. Calculator.vi is the AppDomain where the VI itself is running.

Another important thing to note is the Load() and LoadFrom() assembly bindings that are going on. If you remember from my previous post, LabVIEW uses both techniques to pull in assemblies and you can see that here. Anytime you see the regular, full name of the assembly in “Description”, you know that you are dealing with a Load(). If you see either WhereRefBind or something like “file://” you are dealing with LoadFrom().

So, here is a good example of failure followed by success. Notice that in LabVIEW.exe there is an attempt to load DNDomain, Version=1.0.0.0. etc. etc. If you double click on that, your browser will pop up and display something like

*** Assembly Binder Log Entry  (3/24/2005 @ 9:26:41 AM) ***

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:  C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\fusion.dll
Running under executable  C:\Program Files\National Instruments\LabVIEW 7.1\LabVIEW.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: DisplayName = DNDomain, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
 (Fully-specified)
LOG: Appbase = C:\Program Files\National Instruments\LabVIEW 7.1\
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = NULL
Calling assembly : (Unknown).
===

LOG: Processing DEVPATH.
LOG: DEVPATH is not set. Falling through to regular bind.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Post-policy reference: DNDomain, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
LOG: Attempting download of new URL file:///C:/Program Files/National Instruments/LabVIEW 7.1/DNDomain.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/National Instruments/LabVIEW 7.1/DNDomain/DNDomain.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/National Instruments/LabVIEW 7.1/DNDomain.EXE.
LOG: Attempting download of new URL file:///C:/Program Files/National Instruments/LabVIEW 7.1/DNDomain/DNDomain.EXE.
LOG: All probing URLs attempted and failed.

So, we have a failed load attempt. This is what happens when we tried to load in one of our assemblies (DNDomain.dll) using Load(). However, notice that there is a second attempt in LabVIEW.exe to load dndomain.dll (look at the WhereRefBind’s). If you open that one up, you get

*** Assembly Binder Log Entry  (3/24/2005 @ 9:26:40 AM) ***

The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.

Assembly manager loaded from:  C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\fusion.dll
Running under executable  C:\Program Files\National Instruments\LabVIEW 7.1\LabVIEW.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: Where-ref bind. Location = C:\Program Files\National Instruments\LabVIEW 7.1\resource\dndomain.dll
LOG: Appbase = C:\Program Files\National Instruments\LabVIEW 7.1\
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = NULL
Calling assembly : (Unknown).
===

LOG: Processing DEVPATH.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///C:/Program Files/National Instruments/LabVIEW 7.1/resource/dndomain.dll.
LOG: Assembly download was successful. Attempting setup of file: C:\Program Files\National Instruments\LabVIEW 7.1\resource\dndomain.dll
LOG: Entering run-from-source setup phase.

Here we have a successful load. Again, our edit mode code first tried to load DNDomain via Load(), which failed, but our assembly resolver stepped in and loaded it via LoadFrom(). This is why it is important to always show success and failures when debugging load problems.

Debugging Load Problems

The key to debugging an assembly load problem is to understand the rules of Fusion and see what Fusion tried to do in loading your assembly. It really is just that simple.

The most common problem for people using LabVIEW is just the inconsistency between edit mode and run mode (which is fixed in Constellation) and the log viewer can quickly show you that difference. Looking at where Fusion is probing for your assembly tells you where you should keep it.

If you are getting a more exotic failure, such as access denied, bad image, etc., Suzanne Cook has some good advice here, here, and here. And with .NET 2.0 coming out, don’t forget this.

Custom Setting

Some of you more observant people may have noticed that my logger was configured for Custom instead of default. That is just because I have set the log directory for the viewer. Out of the box, FUSLOGVW uses the IE cache to hold its log entries. You can, however, set the LogPath string value in the registry Fusion key (see above). In this case, Fusion puts the log entries to the path specified and you must click the custom radio button to view them.

NABAMSJBBTWJJ

What? you say...It is just an update on Not as big as medium sized Jock but bigger than wee Jock Jock, our current foster.

The new pictures are up on the site today, but the great thing is that his personality is coming through. He is definately a person dog - follows me around, leans on me and loves to flop over for belly rubs. But get him outside near a tennis ball and be prepared for a lot of throwing. He just loves it.

More info on those crazy Assembly rules

If you have read my posts on the loading of assemblies in LabVIEW, you might remember me talking about the differences between Load() and LoadFrom(). Well, here is a nice little article that goes into more detail on the seperation between them...

http://www.gotdotnet.com/team/clr/LoadFromIsolation.aspx

So, what is a .NET application?

Check it out for yourself: http://pluralsight.com/wiki/default.aspx/Keith/ManagedAppSurvey.html

And second in the lineup...

Well, it is time for the second try at fostering and this time we landed an actual Sheltie (see stories of Lizzie Godzilla NotASheltie)...It is a medium size male somewhere between 2-3 years is the current guess.

It, you say? Well yes. We just picked him up from Town Lake Animal Shelter where he has been neutered. Between the pain and the drugs I don't think his true personality is coming out yet. I've always preferred to name the dog once we get to know them, so he is currently John Doe.

Actually, I've been calling him Jock, as in "Not as big as medium sized jock but bigger than wee jock jock" for you Terry Pratchet fans out there! We will see if that sticks.

So far however I have to say he is a complete sweetie. Very friendly and not scared of people or dogs. We should have him up on the website soon.

C# Open Source Projects

Saw this from ServerSide.NET...check'em out.

http://csharp-source.net/

Psst, can you keep a secret?

DISCLAIMER: All discussion regarding upcoming releases of LabVIEW are not commitments that the final product will be as I describe here. It’s merely what we are thinking at the moment or have implemented to date. Of course, if you have any feedback on this, please post it up or contact me. That is part of the reason for these posts.

So, we’ve talked about assemblies. We’ve talked about how LabVIEW 7 does things. Now it’s time to gaze into the crystal ball and take a look at the upcoming release: Constellation.

I’m letting the cat out of the bag a bit on some new Constellation features to explain everything, but I’ve been assured by marketing I won’t “disappear” after this.

New and Improved

To keep things simple, I’ll start with how things work using LabVIEW in the same way as before (versus projects, which we’ll discuss next). In this case, we cleaned up the algorithm to make it a bit more friendly and robust.

First thing - that Assembly Reference Window is gone! We’ve integrated the .NET nodes into the LabVIEW linker. This means that in addition to the full name of the assembly, we also store the relative path inside the VI (for non-GAC assemblies). Not only does this give you more flexibility, but it also means that if LabVIEW can’t find the assembly, it’ll prompt you with the file dialog to locate it. Nice, huh?

The next thing is that we’ve made the edit mode and run mode behaviors consistent. The algorithm always attempts to load the assembly via a Load() call. If that fails, and we’ve a path, we can fall back to LoadFrom(). This means it’s still best to keep your application and assemblies together, but as long as the files stay in the same relative place during a deployment, it’ll continue to work.

You might wonder why we don’t do a LoadFrom() in the first place if we’ve a path? Well, the answer has to do with how Fusion works in 1.1 of .NET. One key difference between Load() and LoadFrom() is that the first one obeys the .NET version policy files, while the second ignores them. To give you access to the full power of .NET, we are stay in line with the recommended approach as much as possible.

But wait, there’s more

Yep - that’s right. Constellation introduces the concept of projects into LabVIEW. They make it so much easier to do LabVIEW development that I think people will forget (or try to forget) what it was like without them … remember life before Undo?

In a lot of ways, projects are very similar to the project/solution paradigm that you may have seen in Microsoft’s Visual Studio but with some extra twists thrown in for the world of LabVIEW. Well, there are a lot of neat things about projects, but I better stop here until a beta version goes out - I really don’t want to know whether the trunk light goes off when the lid is closed ….

Anyway, the biggest change that projects introduce for .NET integration is that we moved the first-class citizen award from top level VIs to projects. Thus, instead of creating an AppDomain for each top level VI, we create one for each project (actually one per computing machine per project, but we’ll save that for another post - you’re gonna love projects!).

This has one interesting side effect: currently if you do have multiple top level VIs, you can’t share .NET refnums between them, nor use any sort of static state of a .NET class, without a whole lot of work. But now, since all top level VIs in the same project are in the same AppDomain, this works fine.

Also, the AppBase of the domain is set to the home directory of the project. Does this change your deployment? Not necessarily. While projects do let you have your VI’s scattered all over your hard drive if you want, I think most people will keep their main application VIs in same directory as the project they belong to. If that is the case, nothing changes.

What if you don’t? What if your .NET VIs are more of a library shared among different projects? No problem - just keep the relative location of the library VIs the same, and the linker will resolve the assembly – or you can put them in the GAC.

Julienne Fries?

Another change that impacts our assembly view is a new feature -- .NET Controls. Yep, you can now drop a .NET WinForm Control onto a LabVIEW front panel, just like you do with ActiveX controls. Boy - this one was a lot of fun for the team let me tell you. Getting .NET events to flow into a LabVIEW callback was an interesting challenge. But it works!

This actually doesn’t impact the assembly stuff as much as you might think. As with any assembly, you need to follow the rules described above for your controls, but that’s about it. The only real change is that it introduces another AppDomain.

What, another one? Yes, but don’t worry about this one. It’s simply the AppDomain we use to display the .NET controls on the front panel when the VI is not running. This allows you to access the edit mode properties of the control, which still require some .NET code execution.

Spaceman signing off

I figure that is enough for now. I hope you are going to be very pleased with these improvements, as well as all the other new features in Constellation.

Let me know if there is anything else you would like to hear about. It can be either an expansion on something we already discussed, or something totally new – you decide.

LabVIEW 7 and Assemblies

Okay, so you want to know about LabVIEW and assemblies, huh? Well sit back and get comfy.

DISCLAIMER: For the most part, I am going to avoid assuming that you are a .NET or LabVIEW expert. However, I am not planning on going into great detail on things not directly related to the post. If you ever want more details on something, please post a feedback and I’ll see what I can do.

What is a Top Level VI?

Basically it is the VI where you clicked the run button. All the VIs that it calls are its children – ergo they are not top level. However, while that one is running you could go and hit run on a different VI and presto – you have two top level VIs.

How does LabVIEW load assemblies in 7.0/7.1?

In the first version of our support for .NET, we created multiple AppDomains depending on the situation. Basically, you got one AppDomain for your edit-time work inside the LabVIEW environment, and one for each top-level VI that ran.

What about assemblies? Well, since each AppDomain is effectively its own process, each one must load its own assemblies. This is where the fun starts…

Within the VI containing the constructor node, LabVIEW saves the fully qualified name of your assembly. For example, if it is an unsigned assembly then you get

ClassLib, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

If it is a signed assembly, then you get

System, Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

This is all that is stored. When it comes time to load the assembly, if it isn’t in the GAC and it isn’t in the same directory as the top-level VI, LabVIEW will fail to load it.

But what about the .NET Assembly References that you put into LabVIEW? That little database of assemblies under Tools->Advanced? Well, truth be told, we made a mistake. We have some very different execution paths depending on whether you’re in edit mode or run mode.

During edit mode, we look at that list of assembly references to resolve any attempts to load.  We first attempt to load via LoadFrom() with those references and then fail over to Load() if that does not work.

During run time, however, one of the first things we do for either a constructor or a static method is attempt to load the assembly via Load() – oops. If that fails, you get the classic:

“Error 1172 occurred at File or assembly name ClassLib, or one of its dependencies, was not found”

So, for 7.0 and 7.1, please put your assemblies either in the GAC or in the same directory as your top-level VI.

Yes, but what about that version problem discussed on the forum?

Actually, I believe it is the problem we just discussed. I hope to get in touch with Ratin and AnalogKid2DigitalMan to learn more, but their talk about updating the references indicates to me that the updated DLL lived in a new directory. This means that it most likely fell into the problem area above. The reason the error message referenced the old assembly version was that that version was what was stored in the VI. If the old DLL had been overridden by the new DLL, LabVIEW would have gone on its merry way.

Golly gee, Spaceman, what’s next?

Ah, for that you must wait for the next exciting episode of “ASSEMBLIES IN SPACE!!!!” Watch our intrepid heroes as they visit the nearby Constellation…

LabVIEW - Background on Assemblies

A month ago I ran across a discussion on the NI DevZone forum. I though, “Man, I gotta say something about this on my blog”. A month ago? Well, I’ve spent the last month trying to get my arms around the LabVIEW .NET code base. So there.

… Okay, okay, yes, I was playing Half-Life 2 and KOTOR II, but what do you expect?

Anyway, back to topic…

I’m starting with some background material on .NET. If you already know what AppDomain and Assembly are, how assemblies are loaded, and general rules on Fusion, then you can skip to the next blog entry.

What is an AppDomain?
You can think of an AppDomain as a process that lives in another process. Not good enough? … read on.

You see, it is very expensive to create a process in Windows. Under Unix – before multi-threading came about – you did MT programming by spinning off new processes. Simple and lightweight was the goal.

The world was different when Windows NT arrived on the scene. This world had multiple threads, so processes could be as expensive as you liked.

But over time, Microsoft realized that this model had its own problems with performance. When .NET was being designed, they decided they wanted the ability to host multiple applications together in a single process. But if you were around for Windows 3.1 or pre-Unix Mac, you know how ugly it can get if you have different applications all running around, able to see each other’s memory … excuse me a moment, I’m going to be ill …

The solution in .NET is an AppDomain. It’s the boundary box that’s put around each of these applications in the process. The best example of AppDomains is ASP.NET – where each web application (virtual directory) you create is hosted in the same process (aspnet_wp.exe on Windows 2000 or w3wp.exe on 2003/XP) but gets its own AppDomain.

Okay, that was the simple version. For the complete version, check out Don Box’ book on Essential .NET.

What is an Assembly?
Okay, this one is much easier. The simple definition: complied .NET code in either a DLL or EXE format.

In truth, an Assembly is a bit of a logical concept because you can have multiple DLLs and resources (known as modules in .NET-speak) that make up a single assembly, but this is actually kind of rare. If you just think of any .NET EXE or DLL as an assembly, you are 95% of the way there.

How does .NET load assemblies?
In an attempt to get away from “DLL Hell” – where DLLs were dumped in the system directories, order of paths affected what got resolved, and updating one app broke another – .NET defines an exact process for loading assemblies.

First – all assemblies are versioned. When you create your assembly, you assign it a version of the form <major>.<minor>.<build>.<rev>.

Second – assemblies have the option to be “strongly named” (aka signed). This means that not only do you have a version, but you also have digitally signed the assembly (run it through a cryptographically-strong hash algorithm to ensure it isn’t tampered with).

Third – if you do have it signed, you can put it into the GAC (global assembly cache). Okay, yes, this is a bad name. It isn’t a cache at all but a repository. Everyone knows it’s a bad name, let’s move on.

When an application asks .NET to load an assembly, it has a few options which eventual boil down to Assembly::Load() and Assembly::LoadFrom().

Load()
The first case is the preferred one and allows .NET to find the assembly according to the rules of .NET. The simple version of these rules is

1. Is it already loaded?
2. If not, and it is a strongly named assembly, check the GAC for that specific version. Also check the version policy files to see if we should upgrade to a newer version.
3. Still not found it? Try looking in the AppBase of the AppDomain (more later), ignoring version.
4. Still not found it? Okay, check the specified sub directories of the AppBase ignoring version.
5. Still no? What the…fine, I quit.

The AppBase is sort of like the old “current working directory” for normal applications. But remember that a .NET process can have multiple AppDomains, so what exactly would the CWD be then? Since CWD is a bit restrictive, each AppDomain gets its own “home directory” known as the AppBase.

You can also tell the AppDomain what subdirectories to look in. For example, ASP.NET lets you put your assemblies in a “bin” subdirectory of your web application’s directory.

LoadFrom()
It may be that you don’t want to play by these rules. After all, they are pretty restrictive. Either it lives in the GAC or it lives somewhere under your AppBase. Personally I like the new rules, but there are times when they don’t provide everything you need (like in LabVIEW’s case).

To the rescue comes LoadFrom(). This method lets you specify the exact directory location of the assembly. Just like the good-ole Win32 LoadLibrary(). However, this is not without its problems, as you can read in detail at Suzanne Cook’s blog.

Thus ends the lecture
That wraps it up for this post. Next up – LabVIEW vs .NET in the WWWF knockdown!