Avoid collisions between parallel installations of Cygwin

[Larry Hall (Cygwin Developers)]

On 10/15/2009 06:40 AM, Corinna Vinschen wrote:
> On Oct 15 02:18, Larry Hall (Cygwin Developers) wrote:
>> On 10/12/2009 12:28 PM, Corinna Vinschen wrote:
>>> Hi,
>>>
>>> this is sort of a follow-up to a mail from Earnie:
>>> http://cygwin.com/ml/cygwin-developers/2009-05/msg00042.html
>>
>> <snip>
>>
>> OK, I realize I'm chiming after allot of discussion has already taken place
>> but I figure better late than never. ;-)
>>
>> <more snip>
>>
>>     However, perhaps the more interesting thing is to explore the
>> possibilities if the
>>     answer is "No".  What else could we do?  Well, since this implies we'd
>>     need to make 1 DLL work for everyone, we'd need to make it easier for
>>     everybody to use that DLL.  This has been discussed some in the past
>>     but I think the key point would be that there would need to be an easy
>>     way to find and update the installed DLL.  I think if we had some kind
>>     of registry (yeah, I guess we could even use the Windows one if we had
>>     to ;-) ), this would solve the "where do I look" problem.  Obviously
>> there's a
>>     trade-off to this as well, since the user is still limited to one DLL.
>> But it
>>     would be an approach that would allow 3PPs to install in a compatible way
>>     with Cygwin and vice versa.  I expect there are some other possibilities out
>>     there that could serve as a solution in the "No" case too.
>
> I think I see what you're up to.  Whether or not we're having a solution
> for the shared object collision, we have always the problem of multiple
> Cygwin DLLs anywhere on the same machine, give that we always have third
> parties using Cygwin for their own product.
>
> So, wouldn't it be nice if every Cygwin DLL on the system would add some
> system-wide available information where it can be easily found?
>
> Let's simply assume as you did, that the Windows registry is the right
> place to do this.  Assuming further the DLL creates a 64 bit key value
> (heard that before?) from its installation path which is used for the
> object names.  It could now simply create a new registry entry
>
>    [HKLM|HKCU](*)\Software\Cygwin\location\$installation_key,
>    type REG_STRING, content = $installation_root.
>
> (*) depending on user rights, obviosuly.
>
> Cygcheck could fetch this information and print it.  Even without
> cygcheck it's easy to examine the registry and check if there are
> multiple Cygwin DLLs installed.  All past and present installation
> paths can be fetched from the registry.

True enough.  There would be some benefit to this even with the status quo.

>>   2. Now assuming the Devil is sent home unfulfilled, ;-) I've been doing some
>>     thinking about what multiple Cygwin DLLs in their own little worlds
>> mean.  Or,
>>     perhaps more importantly, what kind of problems that we haven't seen before
>>     because we always stopped at the "1 Cygwin DLL" door will surface now that
>>     the door as well as the whole side of the building is gone. ;-)  I know
>> this is
>>     an incomplete list but here's a few things I thought of:
>
> We never really stopped at the "1 Cygwin DLL" door for a long time.
> Multiple Cygwin DLLs running alongside are a fact at least since
> the creation of MSYS.  Nobody has been kept back to create a similar
> solution from later versions of Cygwin and I know for a fact that
> this already has been done.
>
> By creating our own, builtin solution for these scenarios we have now
> more control of the solution.  There won't be a reason for anybody to
> hack something up by themselves.  Well, sure, except for the reason of
> adding some sort of disguise.  Still, Cygwin is OSS and the solution
> is definitely not rocket-science, so you have no way to prevent that.

I completely agree that it's better for us to provide a solution and present it
for use than have multiple external sources take their own shots at it. And, of
course, you're right that we can't stop anyone from doing whatever they want,
no matter what we do.  We've seen that already, as you've pointed out.  But
I think there are 2 points here that are worth reiterating:

   1. Without some provided solution/policy, people are inclined to provide 
their
     own.  Or, perhaps more realistically, if we provide something to address a
     need here, folks will more likely "buy" than "build", since that's 
easier.  Some
     "build" now because they don't see a "buy" option.

   2. While it's true this door has been opened by some in the past (MSYS is 
the
      obvious example), I still believe this isn't a commonly exercised mode of
      operation for most Cygwin users.  The 3PP I've seen simply deliver 
cygwin1.dll
      (unfortunately) and I expect most would switch away from this if they were
      given another option.  So I expect that whatever solution we come up
      with will increase the number of possible cases for multiple Cygwin's 
allot.
      Side-by-side 1.5/1.7 installations are one area of this too, though I 
don't think
      that's going to be our typical problem vector on this list because I 
believe
      the number of side-by-side installations will diminish over time after 
1.7 is
      released, those who have it typically "get" the idea of separate 
environments,
      and it's not that unnatural to view the two installations as separate, 
especially
      since we've been talking about it this way for a long time, even 
before support
      for this was added.

>>     A. A user has a Cygwin installation and the "Nifty-Difty Ultimate SSH"
>> 3PP installed.
>>       Surprisingly, "Nifty-Difty Ultimate SSH" hangs for some reason (it has
>> a bug - who
>>       knew this could happen) and the user tries to kill it using Cygwin
>> tools.  The user
>>       sends email to the list asking why he can't find and kill the
>> "Nifty-Difty Ultimate
>>       SSH" process using Cygwin's 'ps' and 'kill' (not '/bin/kill -f').
>
> Sure, this could happen.  The error message is probably
> "kill: XYZ: no such process".  And `ps' reveales that ssh is not
> in the process list.  `ps -W' shows that there's a ssh process
> installed in "C:\Program Files\Nifty-Difty Ultimate SSH\bin\ssh.exe".
>
> I don't quite see the problem.  Many "multiple Cygwin" problems on
> the list started out with an error description which wasn't very
> helpful, unless, in the second or third reply, there was the hint that
> the tool was installed from the "Nifty-Difty Ultimate SSH" site.
> And it was very certainly not always combined with a simple "multiple
> Cygwin" error message.

True but it was obvious to ask for a 'cygcheck' and to tell the poster to
look for multiple DLLs.  And the message was there for the user to indicate
the problem and a possible solution.  Of course, the scenario above wouldn't
happen currently.  The multiple DLLs complaint would either show up when
trying to run or the 3PP would "just work".  So I see this as one of multiple
potential new vectors that we will start seeing.  Will this occur more than the
current situation?  Probably not (otherwise, why bother! ;-) )  But my point is
that if we're going down the road of supporting multiple DLLs in their own
environments, there isn't anything right now that I know of that helps the user
understand what's wrong (not even a cryptic error message ;-) ) if something
goes wrong and we're going to need to develop more diagnostics and/or 
procedures
to help resolve these.  This could still be an overall win, if the number of 
issues
on the list is reduced and the aggregate time spent on them is less (over
the long term).  I'm guessing that's what you expect.

>>     B. A user has a Cygwin installation and a 3PP with a cool new command-line
>>        tool that filters stdin.  User dumps the output of a file using
>> Cygwin's 'cat' and
>>        then pipes it into the new 3PP command-line tool and gets nothing.  The
>>        user contacts the list to ask why.
>
> There are two possible outcomes.  Either the Cygwin DLLs are happy with
> each other since they are running basically the same version of Cygwin.
> In this case the above problem just doesn't occur since piping still
> works as expected.

Yep.  Thanks for pointing this out.  It is an important point that I glossed 
over.
Of course, if this is one of those times when the 3PP needs it's DLL or Cygwin's
'cat' uses something new in Cygwin's DLL, this could still be a problem.  And
although we run that same risk now if the 3PP delivers their own DLL, if we
change the policy to allow multiple, coexisting DLLs, there may be a perception
that we should allow this as well.  So we'll have to be prepared for that. 
Again,
besides the possible perception change, this doesn't seem like a new issue 
really.

> Or, the DLLs differ in some respect which is catched by the "multiple
> Cygwin" test, and you'll get the "multiple Cygwin" message.

Ah, that's good.

> Either way, no harm is done.  Either it works, or a problem is detected.
>
> Just to be sure I tried this in the test environment created for our
> customer.

OK, so this one doesn't seem like it's much of an issue, which is good. :-)
Thanks for checking this out.

>>     C. A user has a 3PP file browser.  After subsequently installing Cygwin,
>> the
>>        user sends email to the list asking why Cygwin's view of the file system
>>        doesn't match that of the 3PP file browser.
>
> Isn't that really easy to answer?  If the view of the filesystem
> differs, it's very likely that the different tools are looking from
> different perspectives.

Certainly.  The point is that it is another vector for complaints, however 
small the
number may be, all stemming from the change to allow multiple DLLs.  This is a
change from the current behavior where the DLL is either just shared and see the
same thing or the DLLs collide and a message is issued.  Obviously, we can 
answer
this question and provide the user with options but there's not much there 
to help
the user understand why this is the case, why it doesn't "just work" or 
doesn't work
when it used to (of there was a 1.5-based version let's say).

This isn't an unsolvable problem and I'm not suggesting that any of these 
issues and
others that I haven't mentioned are.  I just want to point out that there 
may be a bevy
of end-user issues that flow from this change, even while it addresses an 
annoying
and real issue.

>>     there's a potentially more important
>> issue of how
>>     are we going to be able to diagnose these issues so that we know these are
>>     "you're using multiple Cygwin DLLs" issues and not some other bug in Cygwin
>>     that needs addressing?  Certainly it seems to me that we need some
>> augmented
>>     tools (cygcheck?) to help us understand how many different DLLs are on
>> the user's
>>     system and which ones are in use at least.  Along that line of thinking,
>> it seems to
>>     me that we could make use of a (the?) registry again to keep track of
>> all the
>>     Cygwin DLLs installed on a system and even which ones are loaded in
>> memory.  I'm
>>     thinking here of the Cygwin DLL itself managing these entries, but I
>> don't want to
>>     get too bogged down in details of this one diagnostic feature when I'm
>
> I think your registry idea is a really good one, and it isn't even
> necessarily only a good one with my patch.  An easy way to detect all
> (well, most, maybe) Cygwin DLLs installed on a system would be a cool
> thing, since checking $PATH is not exactly bulletproof.

Yeah, I was thinking the same thing.  So doing something along this line 
probably
makes good sense, no matter what we end up doing about multiple Cygwin DLLs.

>> really more
>>     concerned about any others that I most certainly haven't even dreamed
>> of.  Has
>>     anyone started enumerating a list of "possible problems" that we may see
>>     popping up on the list if we allow multiple, disjoint Cygwin
>> environments in the
>>     wild?  Should we be trying to think of the potential problem areas so
>> that we
>>     can assure ourselves that we have a good chance of being able to classify
>>     them and offer responses and solutions (or policies) for them?
>
> As I noted above, the potential problems aren't really new, given at
> least the existence of MSYS.  MSYS exists for many years now.  Have you
> searched the list for problems which have been reduced to the
> co-existance of MSYS and Cygwin on the same machine?  I didn't, but then
> again, I can't remember any of them.  If there are some, they are very,
> very seldom for sure.

Yeah, I don't recall allot of complaints reported to the Cygwin list and it 
sounds
like Earnie isn't aware of anything on the MSYS side.  I think that does 
provide
some support for the idea that this kind of change isn't going to make large
waves in the user community.  Of course, no guarantees. ;-)  And we also have
to consider the differences in the size and clientèle in these two communities.
I'd wager that the typical MSYS user is not a newbie trying to learn about Linux
or someone that just wants to check out a cool new packaged app.  I'm betting
we will see more users of this ilk on the Cygwin side and that group is the 
group
that's more likely to be bitten by any issues that pop up and end up on the 
list.

> So, here's your next question:  Given that there are no or so few
> problem reports which could be tracked down to a MSYS/Cygwin
> co-existance problem, does that mean the potential problems are
> exaggerated, or does that mean we just didn't detect the cause of the
> problem?  As you can easily guess, I have no answer to this question.

It's a reasonable question and there's no way to really answer it beforehand.
I'm not really suggesting that we should try to qualify the potential problems
either.  My concern is that we make an attempt to quantify them as part of
this discussion.  I see this as an exercise that does 2 things:

   1. Make sure we haven't overlooked something really important.
   2. Make sure we have what we need to diagnose these issues.

Yeah, this doesn't sound very easy or like allot of fun. :-(  It just 
occurred to
me that if we're going to make a big change in policy, it makes sense to
understand the new problems that can arise.  This is something that you may
have done allot of already, Corinna, since it quite possibly was part of 
what you
had to do to test what you have.  I don't know.  But I think there is some value
in anticipating the user issues here and see if we're ready for them.  I suppose
this could end up being gratuitous exercise as well, if the end result doesn't
unearth any major (or "showstopper") issues or a huge amount of minor ones,
since we'd likely come to the same conclusion over time after fielding (or 
not ;-) )
problems on the list.  However, I thought that this was worth bring up while we
were talking about this change.


-- 
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
216 Dalton Rd.                          (508) 893-9889 - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?