Despite everything that was good about TCI, it eventually stopped being the right place. As a subsidiary of Bell Atlantic, they slowly became more telco-specific. There was already an SS7 project when I got there. There was a project around DDCMP (a DEC serial-line protocol). There was one around AT&T Datakit, which actually had some really neat properties in terms of flow control and such. Both of these were somewhat related to the "CO-LAN" (Central Office LAN) project, which was basically BA's play to provide Real Networking to distributed users via their own central office. Looking back now, it was an idea ahead of its time. This was before DSL and cable modems. It was after ISDN had been standardized, but before it had really made it into the world. So the project failed because the technology wasn't there to support it, but it's still pretty amazing that in 1989-90 they were thinking about ideas that would still be very relevant in 2020's work-from-home world.
The other direction which led more directly to my departure was TCI's pivot toward network management. Thirty years later, that area is still a graveyard of failed projects, with a few pretentious zombies roaming about thinking (incorrectly) that they can do better. But what galled me about it was that TCI was bringing in a whole new group of people to work on it, who barely even deigned to talk to those of us downstairs who had actual experience with real-world networks and were still making most of the company's money. So I left.
Encore was a very interesting company. They were one of the original "UNIX supermini" companies along with Sequent, Pyramid, and a bunch of others. As such, they were pioneers in symmetric multiprocessing (SMP). Nowadays that's common even in laptops such as the one I'm typing on now, but back then it was pretty exotic and a lot of very basic things were still being worked out. If you know what terms like CAS, MESI, and NUMA mean, folks at Encore are among those you should thank. The system consisted of several types of boards - processor, memory, network, storage - connected via a proprietary bus. I remember handling one of the then-new 64MB memory boards and being very nervous because it was both large and worth more than my annual salary.
My initial task, based on my experience at TCI, was to make the system support DECnet and AppleTalk as well as TCP/IP. For AppleTalk, I simply pointed out that user-space packages called CAP and KIP already existed and worked well, so that was that. For DECnet, there was an interesting story. Encore had chosen to license a DECnet implementation from TCI arch-rival Ki Research, so I got to look at their code. Imagine my surprise when I found some of my own. Even comments I remembered writing, in what was already my trademark style. We eventually figured out exactly who had worked briefly at TCI and shortly after at Ki, but I'm not sure if any legal action was ever taken. You see, TCI (and its parent) didn't care about that product any more.
The other part of my DECnet work was to modify the Ethernet drivers to support a couple of specific features that it needed. One quirk of DECnet was that it needed to use an Ethernet address that was derived from the host's DECnet address, so I needed to add multi-address support based on what already existed for multicast. Then I was basically done and it was time for a new project. At that time we - the "commercial OS" group as distinct from the Mach group that was better known outside the company - were pivoting from a BSD 4.3 Tahoe base to a System V base (R2 when I started but soon thereafter R3). One of our needs was for a properly parallelized network stack, the one shipped by AT&T was supposedly MP-capable but as far as we could tell could never have worked on any imaginable architecture - so that became my main responsibility.
Then another engineer left who had been responsible for nUnix, which was a way to separate out which parts of a UNIX process would be inherited by a forked child - address space, file descriptors, permissions, etc. If this sounds familiar, it's because Linux later adopted the same idea - characteristically, claiming it as original instead of giving credit to predecessors at Encore or elsewhere. So I ended up being responsible for various parts of the scheduler and virtual-memory subsystem to support that. Later we also needed to support a "guest operating system" capability so that hard-real-time MPX could run alongside UMAX (our version of UNIX) with dedicated resources. In a way it was an early form of virtualization, and I became responsible for various bits related to that as well. As other engineers left, I and one other fairly junior engineer also became responsible for practically everything out in user space. Yeah, it was getting to be a lot - especially for someone who still had less than five years of industry experience.
Mentioning MPX also reminds me of another interesting thing about Encore. Shortly before I got there, Massachusetts-based Encore had supposedly purchased Florida-based Gould. It was celebrated as the minnow swallowing the whale, since Encore was valued at $30M and Gould at $200M with corresponding disparities in headcount and so on. It quickly became clear that Gould's priorities and culture were going to swamp Encore's due to sheer size, but it took many years before I heard the real story. You see, Encore never really bought Gould. Gould had already been bought out by Nippon Mining Company. This raised some eyebrows among Gould's DoD customers, who insisted on US ownership. To get around this, Nippon Mining Company loaned Encore the money to buy its own subsidiary Gould. Thus, it was technically a US company again, heavily indebted to - but not technically owned - by a Japanese company, so DoD was satisfied. Nothing had really changed. In reality, Gould had acquired Encore in every sense except on paper.
This is already getting a bit long, so I've broken the Encore story into two parts. The next part will be about the Encore Series 91, a brief detour to Sequoia, return to Encore, and some stuff about Oracle.