Some thoughts on the Java u21 problem

I see all of the platform UI bugs and https://bugs.eclipse.org/bugs/show_bug.cgi?id=319514 caught my attention and I have made several comments on it. I thought I would share my observations and suggestions at this point, being a member of the Eclipse community.

Observations:

1. This problem happens when you run Helios on Windows (possibly other platforms) with the current version of Java (Java 6 update 21). The problem also manifests in the worst possible way, it shows up as a hang (or possibly and infinite loop). As I write this, I’m sure that hundreds of hours are being spent trying to figure out what’s going on by people trying to use Eclipse.
2. If there is such a thing as an emergency about things not working right, I would say this would be it. Downloading the current stuff is broken.
3. This is an Eclipse bug. We are using the internals of software that we should not be, it changed under us and it broke our stuff. Also we did not certify with this software. We need to fix this problem. Complaining about that Oracle changed it and depending on them to fix it is a disservice to our users. There are many many examples of this sort of thing happening (mainly in SWT) and in most cases Eclipse seems to do the right thing by working around the problem and fixing Eclipse so that the people using Eclipse have the best experience possible.
4. The bug happens with many previous versions of Eclipse due to the Oracle VM change, and some solution needs to be provided to them.

Suggestions:

1. Fix Helios right now. Today if possible. Yes this will require a 3.6a or whatever, but we have to make this configuration seamlessly work.
2. I know a lot of work has been done to develop workarounds and solutions for the previous releases. These need to be widely communicated, including on the front page of eclipse.org. We need to get the word out to help people. And we have to keep in ming people will be looking for a solution to a hang.
3. Improve our release certification process. There is a Java release process where they test and and provided releases available for testing and certification precisely so that these problems don’t happen. We are one of the leading IDEs for Java, we should be totally on top of this process (including having relationships internally with Oracle so that mechanism are in place such that stuff does not get broken if necessary). We have the means to see this is coming and we should have prepared for it by certifying Helios on the u21 version of Java. I know we are pretty formal about talking about our reference platforms, we need to do better here.
4. Have some policies about accessing the internals of dependent products. Of course it’s necessary to do this in a lot of areas (particularly SWT and Equinox), but where this is done there should be some extra review and also some more centralized tracking so that we understand these dependencies and can consider this in our certification. Also having the extra review will minimize the temptation to quickly add something that’s an internal dependency. People should try to think long and hard about this before doing it to see if there is another way.

This problem is a very serious problem, and we should do like what they do when they investigate airplane crashes, we should look at this from every angle and get as much learning  out of this as possible and implement improvements of the process at all levels to make sure it does not happen again.

MercurialEclipse and Andrei the “handle it” dude

As I posted previously I was having real trouble with hg and MercurialEclipse.

It turns out a lot of my problems with slowness in decorator updating and startup with Eclipse were related to the number of projects (and maybe size) of my repository. The repo contains 114 projects, a total of 3059 folders and about 98000 files. One project is about 500M, two are about 250M and the rest are pretty small, about half of the projects are only a few K (Eclipse feature projects). About 125 of the files exceed 2MB, 4 exceed 5MB and 1 exceeds 10MB.

I reported a bug on this yesterday http://www.javaforge.com/issue/11928 and immediately Ekke (an hg/MercurialEclipse fan) made some tests and posted them there, Andrei (the MercurialEclipse maintainer) asked several questions and now, less than 48 hours after reporting the bug report, I have received two updates of MercurialEclipse, the second of which reduced the start of time from close to 4 minute to 30 seconds, which is in the acceptable realm.

A while ago I had a similar response from Andrei with a showstopper issue for me which was resolved in less than 24 hours.

I am very happy I made the decision to go with hg, even though it’s a relatively new path from a number of perspectives and the main reason I’m happy about it is that I know that any real problems will be taken care of immediately, and that’s something that’s very rare in this business, no matter how much money you are willing to pay.

Thanks Andrei!

Sadly, converting from Subversion to Mercurial *almost* failed (revised)

(Since posting this I’m more encouraged and I am keeping going, in fact I have now committed to the switch — there are some serious performance problems for my large repo, but given the responsiveness of the HgEclipse folks I expect it will be resolved soon, and I am able to get work done — Thanks Ekke for your comments and encouragement about this)

I really loved the idea of using a DVCS and I settled on Mercurial (hg) because I use a very good bug tracking system called FogBugz and they offered a product called Kiln which provides hosted hg repositories (and good integration with the source control and bug tracking system). I have really begun to hate subversion which is what I have been using the last few years. One of the main annoying things about SVN is issues about copying folders or projects. Eclipse is not smart enough to *not* copy the .svn folders so you quickly end up with a mess if you forget to delete them in the destination. And there are other problems, so I was hoping that hg would be better.

It took tens of hours and many tries to convert my 130 or so projects containing about 1.5G of source and binaries to hg from SVN. This was time mainly spent pushing the newly created repositories up to the Kiln server. Sometimes it would run for several hours and die, or timeout, giving no indication as to what was wrong. And hg push does not seem to push one changeset at a time, unless you tell it to; it’s pretty much all or nothing. So if you are pushing 100 changesets and 90 got completed and it dies, you still get nothing in the target repository. There is also no status indication as to which changesets have been processes. The Kiln folks have provided some useful extensions that make it try to batch the pushes into groups but even this does not work in all situations as the heuristics for guessing the batch size are not good in all cases. Sometimes a single changeset can be huge (I have several big things checked in). I thought this would all get better once the initial work was done.

But that was not the case, I decided to reorganize some large projects by moving them from folders to the top level folder. Pushing to the hosted repo took several hours. And also the Eclipse tooling for hg (version 1.6.0 of Hg/MercurialEclipse) did not seem to figure this out correctly. Even hg seemed to be confused about this (I did an hg mv and then hg commit, but it left behind the old folders).

Another annoying thing was that empty folders from svn were simply dropped in the conversion.

Finally the Eclipse tooling is really not there. When I start up Eclipse with my 130 projects it takes nearly 4 minutes of the CPU being pegged for the hg status to be updated in the project explorer. Compare this with about 10 seconds with svn (yes I have filed a bug about this with the HgEclipse folks — they seem to be very responsive [I have filed other issues which have been fixed quickly]). And it seems to be very slow in responding to changes like ignoring certain files. And this was only after a few minutes of using this with all of my projects. It seems to want to refresh resources quite a lot which is not workable with my amount of code.

And Kiln itself had troubles. I would try to open certain directories using their web support and got a cute message about the Kiln being overheated and they were working on it.

I’m sure all of these issues will be worked out over time, but I have given up after a week of fighting this stuff and am going back to svn. I look forward to trying in a few months when hopefully these issues will be resolved. I am using version 1.5.2 of hg on an Ubuntu system.

CNF Testing for 3.5.2 – Please test your projects

If you are a project that uses the Common Navigator/Project Explorer, please be sure and test over the next week with the 3.5.2 RC2 build.  A few critical fixes have gone into the CNF and we need to know sooner than later if there is a problem:

296253	maj	P3	Linu	francisu@ieee.org	RESO	FIXE	[CommonNavigator] An empty label is not properly shown when it is the only contributed label
295803	maj	P3	Linu	francisu@ieee.org	RESO	FIXE	[CommonNavigator] Source of Contribution set to lowest priority NCE, not the NCE providing the children
296728	nor	P3	Wind	francisu@ieee.org	ASSI		[CommonNavigator] Problem with enablement on navigatorContent extension point
299438	nor	P3	Linu	francisu@ieee.org	RESO	FIXE	[CommonNavigator] CNF viewer state non properly reset when NCEs are activated or deactivated

An incompatibility between Ganymede and Galileo

In working on my company’s product I saw that the correct icons were appearing when the product was run in Ganymede, but not Galileo.  The problem is captured in bug 295803 and has to do with the selection of the Navigator Content Extension (NCE) to provide the label when there are two NCEs that operate on the same content.  In my applications case, it was my app’s NCE and the NCE that takes care of resources.  In Ganymede, the higher priority of these NCEs has the first opportunity to provide the labels, but in Galileo, it’s the opposite, the lowest priority provides the labels, which is certainly not what you want.

I think we should address the above bug in 3.5.2, as it’s a bad incompatible change.

Here is the (amazingly pretty and elegant) code that I used to work around the problem:

        /*
         * Yum. Due to Eclipse bug 295803 we need to change the priority of the
         * NCE for our product. This only needs to happen on a 3.5.0 or 3.5.1
         * system. We sense this by the org.eclipse.ui.navigator bundle version
         * (in which the minor version lags one behind the main Eclipse
         * version.) Note that if this bug is fixed in 3.5.2, then the version
         * check needs to consider only CNF versions 3.4.0 and 3.4.1.
         */
        Bundle bundle = Platform.getBundle("org.eclipse.ui.navigator"); //$NON-NLS-1$
        Dictionary headers = bundle.getHeaders();
        String version = (String)headers.get("Bundle-Version"); //$NON-NLS-1$
        if (version.startsWith("3.4")) //$NON-NLS-1$
        {
            INavigatorContentService ncs = getNavigator()
                    .getNavigatorContentService();
            NavigatorContentDescriptor desc = (NavigatorContentDescriptor)ncs
                    .getContentDescriptorById("com.oaklandsw.transform.navigatorContent"); //$NON-NLS-1$

            try
            {
                Field[] fields = desc.getClass().getDeclaredFields();
                for (int i = 0; i < fields.length; i++)
                {
                    if (fields[i].getName().equals("priority")) //$NON-NLS-1$
                    {
                        fields[i].setAccessible(true);
                        fields[i]
                                .set(desc,
                                     new Integer(Priority.LOWEST_PRIORITY_VALUE));
                        break;
                    }
                }
            }
            catch (Exception e)
            {
                Util.impossible(e);
            }
        }

RCP, p2, Vista and VirtualStore

I have an RCP application that uses p2 and needs to run on all of our platforms (this uses 3.4.2).  Everything was great until Vista came along.  On Vista when going to the Software Updates… dialog it gets the “not properly configured” dialog and won’t let me do the fun p2 stuff.

The problem is that Vista does not like it (sort of) if you try to write things in the Program Files directory.  You can write as much as you like only during the install process, and there seems to be some way that it writes a “virtual store” into this area when the app is running, but p2 will not work with this.

The solution that I have found is to (my app is called “osdt”, which is also the name of my launcher executable):

1. Add
   • -Dosgi.configuration.area=@user.home/osdt/configuration

   to my Launching Arguments in my product file in the VM Arguments section for all platforms.  This tells it to look for the configuration area in a location that’s relative to the Java “user.home” system property (c:/Users/<User name> on Vista and c:/Documents and Settings/<User name> on XP.)

2. Add the following into my config.ini file:
   • eclipse.p2.data.area=@config.dir/../p2
   • osgi.instance.area=@user.home/osdt/workspace

This makes sure that the p2 data area is relative to the configuration and my workspace is also created relative to the user’s home directory.

3. Teach my installer to put the configuration and p2 directories in the expected location. I use install4j – and the key here is to have it come up with the variable setting the location to install these things at install time using the “user.home” system property from Java so that the Eclipse @user.home and where the kit is being installed to will always be in sync.

All of this was great, but it did not work, and I was finding that no matter what I did to my osdt.ini file it had no effect. The problem was that the Vista “virtual store” mechanism had written my information from the previous installs and that was silently overriding what was in the Program Files folder. In general, if you write something to the Program Files folder, it will really be written to c:/Users/<User name>/AppData/Local/VirtualStore/Program Files, and whatever is at this virtual store location will silently take precedence over what is in the Program Files folder. Once I deleted the stuff in the virtual store location everything was fine.

Application “…” could not be found in the registry.

Just spent a couple of hours debugging one of these and thought I would write it up in hopes that it makes things better.

My use case is that I’m calling my application from an API which means I’m dynamically starting it using the EclipseStarter class (with reflection since I can’t reference Eclipse stuff from the JAR file that my customers use).

Everything used to work in this particular set of tests (don’t they all say that?), and then when I run them I get the error below.

It turns out the problem was the plugin (bundle) that contained the application definition (in the plugin.xml) did not get resolved.  It did not get resolved because it depended on a plugin that did not exist.  However there is nothing in the logs that indicated there was a problem loading this plugin, it was just silently not loaded.  I only discovered this debugging the bundle resolution code and wondering why it was not resolved.

So if you get this error and you can’t think of any other reason for it, make sure all of your dependent bundles are present, and ;resoution=optional could be your friend.

This bug has been filed about this: https://bugs.eclipse.org/bugs/show_bug.cgi?id=277058 (this happened in 3.4.2, so maybe it’s fixed in 3.5).

Caused by: java.lang.RuntimeException: Application “com.oaklandsw.transform.runtime.engine” could not be found in the registry. The applications available are: org.eclipse.equinox.app.error, org.eclipse.update.core.standaloneUpdate, org.eclipse.updat
e.core.siteOptimizer.
at org.eclipse.equinox.internal.app.EclipseAppContainer.startDefaultApp(EclipseAppContainer.java:242)
at org.eclipse.equinox.internal.app.MainApplicationLauncher.run(MainApplicationLauncher.java:29)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.oaklandsw.transform.runtime.RuntimeFactory.createRuntime(RuntimeFactory.java:229)
… 76 more