Mischievous Ramblings II

The Joys of Memory Management, part 1

I've been playing with the iPhone SDK recently, and for whatever reason, I'm having much more success wrapping my head around the NextStep programming model adopted by Apple years ago.  Previous attempts were, shall we say, less than thrilling.

My first iPhone app is approaching beta, which means that I think the first version is done and I'm waiting on feedback from someone else before I consider submitting it to the App Store.

This morning I had a wrangle or two with memory management.  That's not surprising, because memory management is one of the bête noires of programming in c or c++, and Objective-c isn't going to be that different.

Or so I thought.

After creating a generally functional version of App#1, I decided to run it through some torture testing.  So I added a few buttons to the UI and wired them up to some test routines.  Essentially, I simulated thousands of uses of the basic functions.  There are two buttons that the user will press and three textfields, so I wired up the test buttons to call each button's target 10,000 times and one to change the value in each textfield 10,000 times and to call the various functions which use those values.

Everything worked, but Instruments told me that there was some serious memory usage going on.  Not a lot of leakage (although there was some), but some serious usage.  Not good. Especially since this is a simple program, without anything fancy going on, except for rolling about 14 million dice.

So I refined my tests a bit, and discovered some stuff worth recording.  Others may find this useful or not, but I know I need to remember this, and writing it down will help do that, even if I never come read it again.

This is a memory leak:

dcsText.text = [[NSString alloc] initWithFormat:@"%d", 15 + (i % 10)];

This is not:

NSString *temp = [[NSString alloc] initWithFormat:@"%d", 15 + (i % 10)];
dcsText.text = temp;
[temp release];

Discussion:

I wasn't actually using the first version, but the second. But I wanted to know if the first version was going generate a memory leak, so I tried it. And confirmed that I knew how the tool was working and that the version I had written was working correctly.  The first version is a memory leak because the allocated string is created and has a retainCount of 1, but it's never released to reduce the retainCount to 0, so dealloc never gets called.  It might be nice if it were an AutoPtr so that it got released when it goes out of scope, but that is a big whopping can of worms that I'd just as soon not open, so we'll let it go at that.  I will, however, write me a few helper routines of the second general form.

NSArray

Once I had established the validity of the above form, I went on to see if I could figure out why the app was: 1) slowing down over time; and 2) using so frickin' much RAM.  I'd been pretty careful, and I was reasonably certain that everything I was allocating was being deallocated, so where was the leak?  Instruments pointed me right at it.  Every leaked block was associated with an Array class of some sort.  I was only using two arrays: one of NSMutableArray and one of NSArray (basically I built the array using the NSMutableArray and then created an NSArray from that so that client code couldn't alter the array).  So what if I just use the NSMutableArray, eliminating the creation of the NSArray?  No help.  What's the retainCount on the NSArray immediately after I create it?  2?  2?  2!

That's not right.  It should be one.  So try releasing it immediately after creation.  Big time crash later in the program.  Obviously something is holding onto it for some reason.  Look at the documentation.  Whenever you use a factory method, the allocated object is added to the autorelease pool.  But the top-level autorelease pool isn't drained until the end of the program, which is why every last one of the 1 million NSMutableArray and NSArray instances are still hanging around.  Obviously I need to add a nearer-scope autorelease pool.  But it's recommended in iPhone programming that you limit your use of autorelease pools.

Clearly that advice is not meant for this situation.  By adding a local autorelease pool where the NSArray is being created and then draining that pool in the same method which creates the NSArray, only one NSArray is kept around at a time, drastically (like, 30MBs of RAM) reducing my memory footprint.  Also, incidentally, locating the spot where I was not releasing one bit of allocated memory, thus the actual memory leak.

The most commonly used function now operates nearly 100x faster than it used to, probably because of vastly decreased memory usage.

Live and Learn.

Work, aka Teaching

Currently I have two part time jobs: TA for an undergraduate business MIS class; and Sr. Programmer for a small game company over town*. Although they seem very different, there is a thread between them: they're both teaching jobs.

The TA job is obviously a teaching job: one day a week I hold three (roughly identical) labs or tutorials in which I demonstrate the basic use of software while the students (mostly) follow along. We've done Word, Excel, PowerPoint, Access, and Visio so far. Most of the material is very basic. In addition I hold three hours a week (over two days) of office hours. So far I've had a total of two visitors, so the office hours are generally an excuse to sit in a room and do my own reading or homework. The last two weeks I've put in 12 hours or so of grading about 46 papers. For all this I get paid (or at least I'm told I do -- I haven't been paid yet) a reasonable amount, considering that I'm a graduate student and I want to find out what teaching is like. This experience is very different from my teaching at PAVI over the summer semester and I value it for demonstrating what more conventional teaching is like.

The Sr. Programmer job is perhaps less obvious, but it's still a teaching job. The small team I'm working in is new and quite young, as is my boss the CTO (OK, he's only young compared to me). My job is not just to create code, but to provide my experience to the younger programmers and to be a kind of security blanket for the CTO, who's quite competent but overworked. So I review designs and help make overall plans (like how the version control system should be used or how projects should be organized). Even though the projects are small (we're doing casual games) it's still nice to be well organized and the experience of working in a more formalized way will be good for everyone when they work on a more complex and higher-stress project.

Before entering graduate school I did a full set of assessment tests this spring with what they call an "industrial psychologist". The tests included intelligence testing as well as aptitude and interest assessments. The top result was University Professor, which surprised me. Although perhaps it shouldn't have, given that both my parents held advanced degrees and my father taught (statistics and applied mathematics) at Columbia, UCLA, and the University of Oregon. I have generally enjoyed lecturing on development and the mentoring portion of the management job. A significant reason I'm taking the MBA is because I need a master's degree to teach at an accredited game development school in the US, and the MBA is therefore "dual purpose" for me.

So this matter of teaching is looming large...

---

*"over town" is local slang for "in Vancouver proper," or so we're told. It's possible that it's an elaborate gag on the newly arrived Americans, but I don't think so. People don't laugh at us if we say it, and we do occasionally hear others use the phrase. And besides, I think Canadians are generally too nice to do something mean like that.

Now It Can Be Told…

I think too few people have read this, so I'm putting it here:

Leaving Adobe FAQ from 2002.

End of The Cow-orker Era

The Daily Cow-orker 23/11/2004 chronicles the actual ejection of The Cow-orker from the office. And the data droppings she leaves behind.

23rd November, 2004.

The Cow-orker may have moved out our office, but like landmines in third world nations, she's the gift that just keeps on giving...

We've been preparing next year's budget, as was the Cow-orker before we finally pushed her out the door in a scene that went something like this:

- HEAVE! -

"But wait, there's more!  I haven't told you everything yet!"

- HEAVE! -

"We'll call if we need you"

"They haven't got an office ready for me yet!"

- HEAVE! -

"Mind the door on the way out!"

"But -!"

- SLAM -

And there was much rejoicing.

But back to the budget.

The budget spreadsheet is a hideous affair with many, many line items.  Our manager places a copy of the spreadsheet in a shared area and asks us each to fill in the lines we're responsible for, so that when we're done the result can be merged back into the original file.

Skip forward two days.  The Cow-orker is now a distant bellow on another floor (and random phone calls throughout the day and flying visits to see if everything's collapsed in her absence yet) and my manager asks me how the budget is going.  When I go to open the file he notices something odd.

"Why are there two copies?  The file was shared so that you could both work on a single copy, and we could merge it back into the master document when you were done!"

"Because, well ... see for yourself."  I open the first file.

He looks at "my" copy.  My information is there, his information is there ... but there's nothing from the Cow-orker.  He starts to ask where the Cow-orker's information is when I open up "her" file.

It contains her information, and nothing else.  Every single other line-item has been deleted, she's rearranged and reformatted the document, and renamed the file with her trademarked misleading and non-descriptive naming convention.  She's also successfully ensured that merging this document with the master won't work and that the only way to get this information back into the master file will be through manually copying and pasting it cell by cell.

He flinches.  "Why would she do that?  Deleting everything else makes no sense at all when we're trying to put together the budget for the whole department.  Why?"

Because sometimes it's hard to find the right words to say "good-bye", perhaps?

The Compleat Cow-Orker. I wish there were a way to send the author a beer.

Wi-Fi while commuting…in England

IPass gets onboard with train Wi-Fi

Wi-Fi sends train passengers rushing to first class

So how come this isn't available in Silicon Valley? You'd think we (they) would be on the forefront of this. Of course, they spent years upgrading track and buying new train cars that it's hard to fit into, so they are perhaps too busy to add useful and cool stuff like this.

My GDC Story Arc

I went to GDC again this year (in conjunction with other family business in the Bay Area). My primary interests were the usual: seeing old friends; making business connections; learning; and shopping.

Oh, and checking in with people I trust as to the reasons why I'm getting good interviews but no job offers. That was really the first priority. I met with people like Gordon Walton, Mary-Margaret Walker (nee Ipser), Ken Humphries, and others.

There was wide agreement that I present a confusing picture to prospective employers. Partly that is because I have so much experience and such a broad range of interests -- I'd be happy to do almost any technical job from grunt programmer to Director of Development -- so long as the project I'm working on is interesting. Partly it's because I spent 4 1/2 years outside the games business at Adobe Software, where I managed a development group for 3 years and worked as a programmer on Acrobat for another 1 1/2. That leaves me with a great deal of technical experience, but none on the current platforms except the PC and Mac (and not much current low-level experience on those).

Another reason is that I do not present employers with the very model of an enthusiastic student whose sole purpose in life is to get a job in the games business. More than one person told me that interviewers really want to hear prospective employees say something like "this is the exact job on the exact project for the exact company that I am looking for. This job will be the pinnacle of my career, and I will be happy doing it for the rest of my natural life."

Homey don't play that*. I am 43 years old and I've spent 22 years in the game and software business. I have been everything from a sole proprietor (lone wolf developer) to a Director of Games Engineering with 25 programmers working directly for me. I've done six startups (depending on how you count) and worked for both one of the largest game software companies in the world and one of the largest software companies in the world. I have created product and ported product, written low-level graphics code and computer opponents, combat systems code, story development compilers and interpreters, UI code, and 3-D code -- in other words, I've done just about everything technical you can do in games, on platforms ranging from the Atari 800 and Commodore 64 through early PCs and specialized platforms like the 3DO M2 up to current PCs. The likelihood that any specific job will be the pinnacle of my games development career is small. The likelihood that any game company will want me to do what I do best (set up development processes and build development teams) for more than a couple of years is small as well.

The solution for this is harder for me. I will not lie during an interview and tell someone that this is the pinnacle of my career, nor will I admit to being anything but a mercenary who's here to do a job and then see if I want to stay on longer. It seems that most companies do not want me on these terms. In thinking about it, however, I am lead to another attitude, from Crash Davis in Bull Durham, "After 12 years in the minor leagues, I don't try out". I need to establish what I want to do and then present myself in a way that makes it more or less impossible to not give me what I'm looking for. So....

I guess I need to decide what I want to do now that I've grown up. After attending a variety of sessions at GDC and discussing the matter with myself, Sara, and lots of other people, I decided that AI programmer or level designer would suit me just fine. In order to achieve that, I should build a portfolio. To that end I've started working with the FEAR code referenced in Alex Champandard's AI Game Development and am resolved to do some level design with one of the interesting FPS (probably Unreal, but I haven't made a hard decision yet).

In the meantime there is work to be done to feed the family, but that's not been impossible to come by, so I'm not too worried, and then there's the whole indie development side of the world, which I'm starting with Torque this week....

That's the story. Not as neat and clean as I might like, but there's a definite plot and a definite goal (OK, several of them) now. It's an improvement.

* I believe I first heard this expression from Ray Greer, formerly of Hero Games, Inc.

Wireless!

I'm on my way to Vancouver BC for an interview. Sitting down at SFO and opening the laptop, I discover I'm in a T*Mobile hotspot. Quick registration, $10 for 24 hours of access (deductible, of course!) and I'm online.

From the airport.

Damn, but I love technology.

Another Stupid Money Saving Idea

Yesterday the Mercury News ran a story about reducing office space in Silicon Valley. The gist of it is that companies are making their knowledge workers work in bullpens (large open rooms without even cubicle walls), "iCafes", or unpersonalized offices (aka hoteling).

Making people who think work in open spaces is stupid. Ever since Peopleware was published, managers have had no excuse for not knowing that leaving their knowledge workers undisturbed as much as possible improves their productivity.

If your phone rings twice a day, you lose two 15 minute periods of work, because it takes you 15 minutes to get back to the mental state of "flow" in which you do your best work. That's 30 minutes lost. If you are in a room with 15 other people, each of those people loses those 30 minutes as well, so that's a total of 8 hours lost. If each of those people's phones also rings twice a day, you're not getting much good work done at all.

So the companies which are doing this in a straightforward and stupid way are saving money on their real estate bills and losing money on the productivity side. And that's not a guess, that's a certainty.

Software and Planning

simplegeek notes a very basic truth that many people seem to ignore.

Software costs time, money, and resources to produce. The more you skip the initial phases of design, the more you have to pay later. I find this idea to be so simple, and yet so ignored by many managers.

Duh.

I entered management in 1997 because I was tired of working for id10ts who didn't know how to build software. There were exceptions, of course, but the broad consensus of the industry seemed to be "think? why should we let our people think? make them work harder! more hours!"

Once I entered management, I discovered that it was a constant struggle against my own upper management to allow my people the opportunity to work in anything approximating "the right way". Depending upon my boss (Greg Gilley at Adobe was excellent — a shining example I have yet to see equalled) I had to fight less or more (sometimes much more) to enable my people to develop software according to something near "best practices". When I switched to Core Technology at Adobe, it became impossible. Upper management was not interested in keeping engineers from burning out — they were interested in everyone knowing that they (the managers) were making sure the engineers were working as hard as they possibly could.

Of course, it's well known (has been for years) that "work harder" is not the way to get the best (or the most) work out of your people. When I hear the stories coming out of EA about how the ROTK project was "managed", I wonder why shareholders aren't suing companies over their failure to use best practices to develop critical products.

Knowledge workers (a quick and dirty definition is "workers who know how to do their job better than their bosses") require quick and active brains to function well. Tired, stressed, overworked brains are not quick and active.

Linden Lab was unusual in hiring me relatively early (they had about 30 employees) in their lifetime to assist them in managing their process. Whether their immediate intent was to make sure people didn't have to much work to do or not I can't say for sure (Philip certainly seemed to have his heart in the right place), but the usual result of organizing process is also some orderliness in decisionmaking about how much work people are going to have to do.

Not A Surprise…

...since I've only been with the company for 13 days, I'm on the layoff list.

I'm OK with that. My nature is more that of a consultant anyway, and perhaps they'll think about engaging me.

I do get a severance package and a lifetime Second Life account (when I have a low-latency connection I can use it on).

Guess I need to talk to people about contract gigs, again.