« Can Rob Novak account for his whereabouts on Wednesday? | Main| Quickies »

Sanity Check


I'm either having a moment of deep insight, or last night's curry chicken was a bit off...

Hypothesis: In normal software engineering efforts, most programming problems are solved by toolkits.  In Domino development, most programming problems are solved by collaboration.

Could that be (at least one of) the reason why it's so difficult for IT departments and software companies (including IBM!) to grasp Domino dev?

Discuss.

UPDATE:  Some continued thoughts on the Permalink...

First off... the curry chicken was definitely off.  Which is why I was up posting to the blog at 3:30am.  So no, I wasn't having a moment of DEEP insight.  But maybe there's some shallow insight in here somewhere that's worth talking about.

Tommy asks what I mean by "solved by collaboration."  Before I answer that, I want to refine what I mean by "solved by toolkits."  In normal software disciplines, programmers work with a wide variety of tools that are provided by their contexts.  If you're a Windows GUI programmer, you start with the Microsoft Foundation Classes and you probably have a dizzying array of add-ons to your source environment.  If you're a J2EE developer, you're probably in a corporate environment with a set list of Beans on some big wall poster that you can work with.  And every interface in and out of them is strictly blackboxed, because you're working with some standardized methodology that tells you exactly what you are and aren't allowed to tinker with.  If you're a php/MySQL developer, you probably didn't start from scratch either.  You probably started with phpBB and customized it, or ran out to phpBuilder and plucked some pieces out of their code library, but odds are you didn't dissect what you got there -- you just stuck it on the Apache server and followed the documentation.

In my experience, Domino developers don't work that way.  First off, hardly any Domino shops actually have tools beyond what ships in the product.  How many of us have ever worked in an environment where there was a standard set of widgets that Domino developers could draw from?  Be honest.  99% of Domino development is done in corporate IT departments, and 99% of those have a fraction of the resource allocation that .NET and J2EE teams get, and 100% of them use less formal methodologies than other dev teams.  How many have you ever seen that take the time to say "we're going to develop a standard tool for interfacing to our product database, then we're going to deploy it to the server and our Domino devs can draw from it?"  Do you think J2EE or .NET teams work in ANY other way?

A lot of people try to blame that on the platform, but that's BS.  It is not hard to build reusable library components that you can plug into your own NTF.  It's not hard to have them update on a continual basis.  It's not hard to have them encapsulated to be blackboxed and reusable.  But how many Domino developers have ever taken the time to learn about cross-template design inheritance?  1%?  How many write Lotuscript custom classes into script libraries that can be easily copied into other templates?  0.5%?

By way of example, I work in an organization with some 15 Domino developers, all of whom are extremely talented and some of whom are world-class.  Yet I have to beg, plead and cajole them into a) drawing from toolkits and b) formalizing their work so that it can be turned into a tool.  And it's been like this at nearly every one of over 30 Domino shops where I've done development.  (Notable exceptions are two product dev teams where I got to run the show.)

How many solution toolkits have ever been brought to market for Domino?  There are developer tools like the Teamstudio suite and Ytria, but these are more about dealing with the limitations of Designer than about providing reusable components to dev teams.  There's a handful of projects on OpenNTF.org that lean towards pluggable components for Domino dev -- Ext.ND, OpenLog, SuperNTF -- but those are the exceptions.  Why don't we see developers flocking to OpenNTF to componentize some of the great work done there?

Why don't Domino devs have a culture of reusability?

So what do we have?  devWorks?  Blogs?  Websites?  Conferences?  These are all collaboration approaches.  The idea is that you learn how to do something, and then you go code it up yourself.

Is that what you do?  It's what I do, all too often.

Part of the issue rests with the approach established by IBM Lotus.  For example, can you identify a single component that is reused across ANY of the templates that ship with Domino?  Feel free to go all the way back to, oh, version 4.  Or how often have we seen IBM sponsored publications on devWorks that result in a description, but not, say, a downloadable piece of DXL that you can import into your own templates?

But most of it is embedded in our culture.  We're just used to solving our problems by asking someone how something should be done.

I think this is reflected in our customer approach as well.  Domino projects tend to be prototype heavy.  They tend to have low governance standards and involve a lot of user-testing rather than QA.  Domino development tends to be a collaboration with the user base, rather than a rigorous adherence to a specification.

Is this unique to Domino?  I don't think so.  A lot of web 2.0 efforts tend to be iterative.  Web sites which are in "perpetual beta" are like that precisely because the testing is done primarily by the user base, and rollout of code changes is quick and easy -- just like on Domino.  But even with the multitudes of web developers out there, we've only recently started seeing real turnkey tools solutions for good web dev, with Yahoo Widgets, ExtJS, Google Gadgets and *shudder* Dojo.

Profound?  Probably not.  But think about this next time you want to complain about IBM's progression on Designer: have you established discipline for YOURSELF to make your Domino work easier?  Do you write for reuse?  The Domino world has rejected just about every attempt by IBM to formalize us -- "Websphere" is a dirty word to most Notes-heads -- but then we gripe that IBM never gives us anything.  Well, they are software engineers brought up in the tradition of formal frameworks.  They think they're supposed to give us toolkits, and we use those to build more toolkits.  Is it really any surprise that they don't understand what we want?  Do WE even understand what we want?

Comments

1 - What exactly do you mean by "solving problems by collaboration"?

I'd be happy to discuss if I knew what you were talking about :)

2 - Collaboration is just what IBM is focused on. Development hasn't a top priority for IBM with regards to Notes/Domino. There are not that many toolkits available because Notes is a proprietary platform which makes it hard for anyone besides IBM to even provide a native toolkit.
My opinion is that neither of these has anything to do with why IT departments or IBM understand Domino Dev.... I'll refrain from ranting on that here... I do that enough on my own site.... Emoticon

3 - This might be right about "no toolkit", but IMO, collaboration is what web2.0 and open source is all about. And thats the problem, behind the "no toolkits": not having things which lets you make such toolkits like you have for Java or other "open source" products:

* No easy/available collaboration tools on source level (cvs, svn, git,...). Think about svn.openntf.org...
* No clean seperation of backend and frontend (think drop in template engine and your own templates) -> How to drop some subforms into your own code without changing all the field-properties to make it look like your look&feel...
* No big userbase, which likes to play with the code: php available is for everyone (how cheap is webspace nowadays?), so is java, kde, gtk,... Compared to that, you have to pay a lot for notes/domino.

4 - It's all a frame of mind; psychological, IMHO. Have IBM release the "Lotus Notes/Domino Toolkit" as a free download for developers. It's really just the Domino Designer client available for free, but all of a sudden businesses will say, "We can now create custom databases that tie in with our existing databases!".

5 - I was thinking about it some more... I really think the reason it's difficult to grasp Domino dev is because there is simply *nothing* like it. You can't even compare it to anything; it was a brand new paradigm, and it STILL is. When there's nothing even comparable, everyone's just at the mercy of having a hard time explaining it. Thus the rebirth of discussion everywhere about the IBM ads.

6 - I think one of the things we are seeing is a specific and deliberate effort from IBM to take Domino out of the 'toy' application space and into the Enterprise space. I think DDE is the major piece to this puzzle. It allows for source code tools, application of enterprise methology and process stuff like Rational, and other items. This is going to take time, and needs to be done in a way that does totally alienate the typical Domino developer. I think by the next major release after the current release under development ... we will see major movement this way.

I think the key here is community education and some bridges being built. I hope IBM listens to figure out what that means.

7 - Darn, I must be in the wrong business. I never realized toolkits were a hard sell in this market. Guess I'll go write .NET widgets instead of Midas or CoexEdit or OpenSesame. And I guess I'll have to break the news to Mark so he stops selling Zmerge after what, 15 years.

You make me so sad.

8 - @6 - John, let's assume for a minute that you're right about wanting Domino to stop being a "toy" application. Why do they think that in the first place? I mean, clearly Domino is a world-class email platform, and email is an application built on the Domino framework. IBM runs their entire internal messaging platform on Domino -- is it a "toy" then?

I think you're right that the institutional generalization within IBM is that Domino apps are toys, though.

WHY?

It's clearly not capability driven. It's not sophistication of solutions. It was once scalability, but not anymore.

That's why I think it's a style thing. It's a cultural believe about HOW enterprise class solutions are built.

There's nothing about Domino that keeps we developers from building componentized, blackbox, reusability-focused enterprise solutions. Nothing at all since at least ND6, and maybe ND5.

But we don't.

Every effort I can remember to even so much as develop a collective set of standards, whether inside a company or across the open community, has basically failed.

That's a cultural phenomenon, not a technical one.

Are we just lazy? Are we trying to inflate billing hours? Are we just too stupid to figure out how to do it right?

I don't buy any of those. There are world-class programming geniuses working in the Domino application space.

But you know this cultural strangeness just as well as I do. You can take the best of the best in the Domino community and put them on the phone or in a forum with the people that are building the next generation of the platform -- and the two sides talk right past each other. IBM says "look, we gave you integrated web services and allowed you use the property broker to exchange information between client contexts on the glass so you could create applications that drew from a variety of platforms and data sources," and we go "but I just wanted computed embedded views!"

That is a HUGE disconnect.

Why?

Understand, the blame isn't just with IBM. They believe they are responding to customer demand. SOMEBODY asked for a Notes client that can consume JSR168 portals via frame-to-frame SOAP transactions. Or IBM thinks they did, at least. But everyone I know who's a Domino dev looks around and goes "who requested this!?"

You know I'm right there with you on the importance of DDE, but in all honesty, I don't think that's going to change the Domino dev culture. And if the issue is really cultural, then we need a bridge. Either the nature of Domino development has to change, and become more formalized and tool-oriented (which will lead to dramatically better, but more expensive and slower-to-market solutions) or IBM has to change what they want to deliver and become LESS formalized and more iterative (which risks breaking the hell out of the Domino codebase.)

Maybe DDE creates an opportunity for give-and-take there, but I don't think we should count on a technological solution to this situation.

Why is it rare for Domino environments in billion dollar companies to have proper change control? Why is it rare for Domino teams of even 10 or 15 to have dedicated test engineers? Why is it rare for anyone to have Domino dev standards?

Nothing about the platform stops these things. We just don't do it. We need to cowboy up and start insisting on using the tools we have to their proper potential.

9 - @7 - Oh PLEASE, Ben! You are the single most successful tool developer in our industry, building perhaps the single coolest dev tool for the platform.

But you told me yourself that your bread and butter is Coex. How many Domino shops USE Midas? If you even say 10% I'll call you a liar.

If Domino developers really used toolkits, you'd be sailing the South Pacific on your yacht, 'cause there'd be 120 million Midas LSX licenses active right now.

Don't tell me I make you sad.

10 - Couple of quick observations before I get yelled at for being late to dinner:

1 - A LOT of Notes/Domino development is done solo, or by relatively smaller numbers of developers than is the case with most other platforms. This is a great strength of Notes which often enough turns into a weakness because smaller teams = less political power = preference is given to the bigger more expensive platform team just because they have more mortgages to pay.

2 - Because IBM in the early 2000s let the perception that Notes was going away persist for so long, few organizations and few developers probably thought there was any point in making reusable Notes stuff if it wasn't going to be around long enough to get reused.

Dinner time.

PS - Thanks for the SuperNTF plug Emoticon { Link }

11 - @10 - You touch on something there that is highly relevant to Domino dev: differences in budget/staffing.

See, I think that's cultural too. You CAN have a Domino team of one or two people that can get a HUGE amount done on the platform. And for the most part, that works. But a proper Domino team should have the same kind of project management, UI design, QA engineering, rollout management and all the myriad other roles that every other platform does.

We just don't do it 'cause we can get away with it.

I've said quite publicly that one of the major vulnerabilities of Domino is that it's too CHEAP. We need bigger budget demands to make the CxO-level radar.

There's a basic cultural barrier that exists in firms, though. Attention is paid to BUDGETS and not to RETURNS. That's a fundamental economic problem with the theory of the firm. We need an advancement on the scale of double-entry bookkeeping to fix that.

Tragically, I don't have Piccolli's genius, so I have no solutions to offer. But I can at least recognize the problem! Emoticon

12 - Nathan -I agree with both you and John.....
John's point about Notes perception as being a 'toy' application is the result of years and years of neglect from both IBM and the dev community within the companies that using Notes. Besides, the way you framed your original question was all about the dev side of Notes/Domino and not the Mail side. So while that may be strong, there is still a long way to go on the dev side.
After thinking about this all day the reason that Ben doesn't have a million licenses is not just because people don't use toolkits - they don't even know they exist! If someone who writes Java apps for an IT organization inside a large company and she/he wants to learn more all they have to do is go down to the local community college or stop at a book store. Where do you go to learn how to be a better Notes/Domino programmer? Who is out there teaching you the basics of software development from a Notes perspective? Most of the IT departments we deal with are not staffed by people with computer science degrees for the most part. That isn't a bad thing necessarily - it is exactly as you would expect for the very nature of Notes/Domino.
People don't use tool kits not because they are stupid - it's because they are ignorant. They are ignorant of good development practices, ignorant of Notes/Domino itself, ignorant of what is even possible - and have no *easy* place to turn to improve. You can't even pick up a magazine that is focused on Notes anymore!

Like you said - the disconnect between IBM and the community is pretty deep. (Here is where the rant begins) We have Composite apps today because IBM didn't get anywhere with Workplace. So now we have Workplace and Portal grafted onto the back of Notes. IBM isn't as focused on improving the development environment or the tools as it is driving the larger agenda. Case in point are the embedded views - you can bet those would have been fixed right quick if it served IBM but they have no issue with it. In fact all they do is bandaid on other shortcuts that very few others could make use of. Almost all the Repids for shipping apps are fixed and the ones that are not have 'magic' repids that always resolve correctly for the user. As far as IBM is concerned embedded views work fine. (rant done)

Your final point is exactly where I want to end also - it is exactly what has to be said so I will repeat it here:

"Why is it rare for Domino environments in billion dollar companies to have proper change control? Why is it rare for Domino teams of even 10 or 15 to have dedicated test engineers? Why is it rare for anyone to have Domino dev standards?

Nothing about the platform stops these things. We just don't do it. We need to cowboy up and start insisting on using the tools we have to their proper potential."

Domino development needs to grow up and that starts with people.


13 - Not enough people recognize the problem.... What are we doing or should be doing as a community to educate people? This is the only reason we launched the governance for notes blog.... to have a place to raise the issues and get more people to talk about it.

All I can say is that I'm excited that you are brining it up too!

14 - Ok... one more thing and I promise I'll shut up.... for today.... Emoticon

Yes I am from Teamstudio, but let me just be the first to say that tools are not the answer. People and policies are the answer. Tools will do absolutely nothing except waste money if there is no commitment to changing - and that begins with recognition that you have a problem.

15 - @13 - Not to be too much a mutual admiration society, but FYI: the governance for Notes blog was on my feeder the first time I saw that existed. Awesome concept, awesome contributors, hugely needed in the Domino space. Keep up the good work!

There's a reason I try to comment on threads there. Emoticon

16 - My $0.02 + interest:

My company has formalized change control, and we have a "Standard Template" for use across multiple applications. The Standard Template is sorely in need of updating, and I've been working on that in bits and pieces.

We try to do things "right", but we are a relatively large team as Notes developers go. Most development efforts, I think, are done by smaller teams.

Yes, proper methodology starts with us, but the very structure and mechanics of Domino are fundamentally different from every other system out there. Domino can't be expressed as code in text running from beginning to end. (Don't say DXL until it's fixed up to be round-trippable.) If someone makes a change to one of your forms, how to you do a diff with the last saved version to see what changed? You don't. I'd give an arm and a leg (well, virtual, anyway) for a Domino version of SVN. The Teamstudio stuff helps, if you can afford it, but it's not quite 100%.

Which brings me to my next point: Commercial add-ons and APIs. After working in a number of Domino shops, I can say two things definitively. 1) It would have been incredibly useful everywhere I've worked, andn 2) No one has ever been willing to pay for it at it's current price point. I'm not going to try to tell Ben or anyone else how to price their products, but I can give a few little hints as to the reasoning, based on my experience.
A large majority of companies:
- are not willing to pay more for an add-on than for the server that you're adding it on to (hello, intelliprint),
- have budgeting systems in place that effectively forbid purchasing software with required yearly maintenance contracts (As one CFO once asked, "Why do they insist that we pay for this more than once?").

I will say that I think that OpenNTF is a good path to getting people to reuse some good code and/or templates. If there were more projects that consist of helpful stand-alone script libraries, you'll see more people using that kind of thinking. As it is now on OpenNTF, most of the projects are stand-alone applications, which don't really lend themselves to having their moving parts dropped into completely different applications without modification.

More as I think of it.

17 - Before anybody gets too worried about my survival, I should point out that I have already sold well over a million licenses, over 1.4 million client licenses last time I checked. When I say that Midas is no longer my bread and butter, which it is not, that just means Genii Software has more successful products still. CoexLinks has been bought by 5 out of the Fortune 10 companies, many of whom don't even USE Notes (they buy it to work with smaller companies they acquire who DO use Notes). People are still buying Midas all the time, all over the world, at its current price point. Honestly, the feedback I get most often is that it is too inexpensive, but hey.

Anyway, the tone here just made it sound like I couldn't hang out in the South Pacific if that is what I really wanted to do. I don't. I like to make toolkits and coexistence solutions and other stuff that people like to buy. Tell me what you need, and I'm interested in making it.

But Nathan is right. That is only about 1% of the Notes users, not 10%, so I guess I still have a ways to go.

18 - @17 - If my influence can put you on a yacht in the South Pacific, I would consider it an honor. Emoticon

That's not a joke.

But you'd better keep a stateroom open for me and the wife.

19 - Thanks, Nathan.

But seriously, part of the reason you all seem to ignore is that it is just so easy to write apps in Notes/Domino. Now, that sounds like a good thing, but easy is often a mixed blessing. Think about an expensive piece of equipment that is easy enough for the boss to turn on, and all of a sudden the boss is going to want to turn it on. IBM gets this, even if we do not. And they have been working on it. I'd say with Composite Applications, they just about have it nailed. Nobody is going to ask Joe down in marketing or Sue up in accounting to create apps now, no sirree.

But the .NET and J2EE folks have a big head start, and their toolkits have been incomprehensible for ages. I've worked in a law firm where lawyers would make their own Notes apps. You think many lawyers are going to start making J2EE apps? But we can catch up. We can make this system hard, even if it kills us. And IBM is on our side.

(Perhaps I had some of that curry chicken too)

20 - IBM still does not want to invest in anything on the Notes platform that can't be sold as non-proprietary which in this case is code for "anyone who graduated from Java U can program it". With the Workplace javascript engine they can expand that javascript programmers. I'm sure that they are relying on the Eclipse IDE platform under Designer.next to goose the tools market. This will help but as Craig said, they're just tools.

Good development practices are self imposed. If IT was return rather than budget driven, they would invest in people. But that would completely blow the fantasy that programmers are generic plug 'n play resources.

21 - @19 That's it! Because it's so easy, Notes has been perceived as an incremental spreadsheet enhancement. Maybe they should have called it Lotus 1234 Emoticon

22 - @19 - Either I just got a major whiff of sarcasm or that curry chicken was more potent that anyone realized Emoticon. That aside, the point about Notes being too "easy" was somewhat implicit in my comment @10.1, but I'm glad you made it again more explicitly. I was once asked in a job interview* what the best and worst aspects of Notes development were, and after a few moments came up with this exact point - that they are the same thing. I've been crusading ever since to try and change that, and SuperNTF is a big part of that.

@16 - Brian, I hear you about needing to update your standard template, and also on the desirability of having more drop-in modules on OpenNTF. I'd love to get your take on my SuperNTF framework, as it is targetted at both these needs. Another design goal of SuperNTF which you might appreciate is to keep the learning curve for understanding the framework as flat as possible. As you probably know well, it can be a real challenge to get developers to use a framework, or use it properly, if it is overly complex. The "modules" in SuperNTF are not as yet clearly documented so that you could readily pick out some pieces you want, but that is the direction I'm heading.

As for Teamstudio, I have LONG advocated that ALL Notes development shops should purchase these tools because far from being "expensive", they are an incredible value. Sure they cost money, but hiring even 1 less developer to manage the workload can leave enough money in the budget to fully equip all but the largest development teams with all the tools. I often use the analogy of building a house, where you could either build it using 80-yr-old hand tools or with all the latest power tools. Would anyone seriously argue in this day and age that spending the money on the power tools isn't cost effective. The productivity multiplier of using Teamstudio tools vs. not doing so is in my experience at least as high as that of my powertool example. Unfortunately, many IT budgeteers are either too narrowly focused on their budget slice or simply don't care to see how lack of tools affects payroll expenses.


*I got the job Emoticon

23 - @16 - Regarding modular projects on OpenNTF.org: the thing to remember is that as an open source community effort, OpenNTF is reflective of its contributors. If there are Domino developers doing modular work, modules will show up on OpenNTF. If there's demand for modules, people will scratch their own itch by contributing.

What the lack of modular designs on OpenNTF tells me is that people aren't doing it. And they're not asking others to do it.

We need some kind of assembler technology to facilitate this stuff. Something where we can take a set of design elements and package them up, and then they can be imported into a developer's own template. DXL will do that (gripes about it notwithstanding.) We just need some kind of packaging interface.

Sounds like a good thing to try to have ready for 'sphere, actually.

24 - Well, if Nathan and I did not believe that some progress was being made than this post would be quite different than it was as written orginally. I think the problem is that while the work on DDE is progressing, sometimes it feels like IBM is building for the top 10 customers instead of the typical one man Domino Developer working in a corner at a company ... providing all their workflow and collaboration solutions. Don't get me wrong ... we need source code control, methodologies that tie to the enterprise world, and more advanced tools. But we also need improvements that help the one-man development shop. We need IBM to follow through on all the efforts they do. No more 90% efforts .. like Rich Text refresh in the front end, Embedded View object not allowing for a soft coded database reference, an incomplete Layer implementation, and many more.

I believe that IBM wants to make as big an impact on Domino Developers with DDE and the next couple versions of Notes and Domino .. just like 8 did for the end user. I just want them to make smart choices that impact every user. Not just Mr. Enterprise Developer who won't do anything without a javadoc and a 20" by 10" architecture on a white board.

25 - I definitely think IBM wants to make a difference. They are making all the right kinds of noises. Part of the problem really does seem to be cultural, and part is the simplicity I was talking about, even if I was a bit obnoxious. The entire Notes development community is very used to simplicity, and IBM is not really geared towards building simplicity. If Notes developers are to use toolkits and frameworks and beans and that sort of thing, they need to be simpler than the .NET, J2EE, etc. versions. That is a lot to ask for, but it is what is required.

Which is not to say that Notes developers won't jump through hoops. Heck, we're practically a travelling circus of hoop jumpers. It is just that the 0 to 60 part has to be incredibly simple to match what we have grown used to, even if the 60 to 100 part is harder than it really needs to be. Most development environments have incredibly difficult 0 to 60 parts, after which 60 to 100 is a comparative breeze, and that is exactly the kind of tool IBM keeps wanting to build for us. Like it or not, we don't have the history or tolerance for that. So, the question is, can IBM front load the simplicity?

26 - @Everyone,

Sorry to be so late to the conversation. I agree with most of what everyone said here. Programming for Lotus Notes and Domino is unlike anything available. The programming model is so disconnected and loose. There is so many different ways that one can approach a problem. As you know you can create a Notes application using so many different languages. However from my experience, you cannot create any decent application using a single language like Lotuscript. You need to know Lotuscript, formula language, JavaScript, and java but without of benefit of a central depository. As result, the controlling the development process control requires discipline, which is not easy if the developers are not well trained which is one of the problems.

Even with formal Lotus Notes and Domino developer training there is no emphasis effective planning and guidelines in developing applications. Lots of the training and books if you can find them still focus on methods that were used in developing applications in version 4. Most developers develop application as single databases even when the database becomes extreme large. When ReCor was creating Domino developer training CBT course, we used the guidelines defined by Lotus and never emphasis best practices and design approaches other than the monolithic approach.

I have had many arguments with others on the importance of breaking up even a small application into small-encapsulated databases to improve reusability and maintenance. It betters reflect the business process. We have been developing using this approach for the past few years. I call this technique, NSOA (Notes SOA). The databases that we develop communicate with each other using SOAP like methods. It takes lots of discipline to program using this approach, but you have a large library small-encapsulated modules that can be reused over and over.

Now for the ranting. In my opinion, a significant part of the blame goes to IBM and their failed Workplace strategy. This reduced significantly the investment that was made to Notes, scared some users to not make further the investment in developing new applications. There are so many little bugs that existed for years and years that they have never fixed even ones that they publicly said would be fixed. You are always almost there when you attempt do something until you will run into a bug or deficiency that has been there for years. The ugly interface could easily been fixed years ago if the resources was put into Lotus Notes and fixing the bugs. So after Workplace failed they shoved the technology into Lotus Notes and now we have a Notes client that is many times bigger than before. Miraculously, we can now do composite applications which you could have done since Notes 5.

27 - Remember IT Factory?

They probably went farther than anyone else in terms of developing a toolkit for Domino developers. The company still exists, but AFAIK the whole framework that they were offering at Lotusphere 1999/2000 was met with a pretty cool reception from most customers.

Were you one of the many people who evaluated the IT Factory offering, but didn't adopt it? Why not?

I think the major obstacle to IT Factory was that by the time they came out with their framework, most customers who could understand the need and benefits were already in mature Notes and Domino environments and just couldn't see themselves going back and re-tooling all their applications to fit into the framework. Customers who were just getting started with Notes and Domino didn't see the cost/benefit of adopting and building with the IT Factory framework.

28 - Richard, you're right. Lotus Consulting also had developed a framework like ITF, and a few companies (mainly in Germany) adopted it. But it never got mainstream traction, for many of the reasons outlined here.

29 - @27 - Rich, I recently had occasion to work on an app that used the ITF toolkit. I was... underwhelmed. But the concept was definitely more in line with modularized, reusable software design.

So you think the reason it didn't take off was that neither existing nor new Domino shops thought it had sufficient value? Well, the kind of goes without saying, doesn't it?

WHY didn't they think it had value? It clearly allowed you to build good, scalable Domino apps much faster and in a consistent way. Why didn't more Domino shops think that was important? Those same IT departments certainly think it's important when it comes to other development platforms, right? Nobody goes to build custom SAP stuff and goes "aw hell, we don't need to plan out a component architecture with reusable modules -- let's just WING THAT SUCKER!!!"

30 - Have you considered that it is the history of the product and the common history of its developers that has as much to do with the way applications get built in the product today as anything else?

To really jump in fairly, I have to start with some history. You can skip ahead if think you already know.

I probably go back further than 99% of Lotus Notes developers, having been certified initially in version 2, before there was any way to get a piece of data from one note onto another other than inherited field values from parent to new child. In those days, nearly all developers of Lotus Notes applications were from the sales and marketing field. They’d put SFA’s and contact managers together on the flight from LA to Chicago and share them across departments through replication. Some of these guys turned out to really “get it” when it came to how people want to work with information. Many became “Notes Developers”.

When version 3.0 came out, and we got @DbLookup and @Dbcolumn, about half those developers fell by the wayside. Around 10% got ahead of the remaining ones and became consultants – doing their magic for hire. In those days, if you really understand how to use readernames, @dblookup, and do view categorization you were a hot shot.

By this time, Notes apps were getting complex enough to be taken seriously as having business value, but to the current I.T. programmer crowd, Notes was already a strange animal. It wasn’t like coding in C, or creating SQL databases. It wasn’t relational, and it made no sense. Only those developers that grew up with it understood it.

By 4.0, the consulting business was bad. Customers had figured out @DBLookup, and were just as good as we were at most things. Then came Lotuscript and the first hint of real programming. Consultants made good money once they learned it, but a large portion of the development community just dropped out at that point.

Fast forward to today, and you’ll find this same crowd still doing the work – but most have never used source code repositories. Most have never written languages which require compiling and linking. Tools like Ant, Link, and Make are scary.

Where would these Notes (now Domino) developers have learned to use these tools? It isn’t where they have been adding value.

No, these Notes Guys don’t work that way. They work in small, neglected teams with less funding using tools that aren’t considered cool. They’re never looked at first for sexy new I.T. projects. They come in the back channel, when a department manager has reached the frustration point with his regular I.T. crowd telling him 6 months and 60,000 dollars. The Notes guys – those geeks who sit alone at the lunch tables in the cafeteria – come in and build it by Tuesday. They just make it happen. Often, they don’t even get formally funded to do it. The app just happens, and the department starts using it.

Until you understand not why, but HOW notes is successful in companies, you’re never going to understand fully the resistance to J2EE, Portal, Workplace, Rational, Websphere, and all things derived from same.

There’s another reason why Notes developers don’t use change control and source management tools. They don’t have to. What I mean is, if you don’t use source control tools, change management tools, check-in/check-out, and versioning on a J2EE project, a C++ project, a vs.NET project, or most other kinds of projects – they fail. The code won’t compile, crashes and dies, or generates nothing but long screens full of stack traces.

Notes Doesn’t Do That. With some very limited exceptions, you can get away with random, shared, development on a database and little more than “Hey, are you still using that view?” kinds of questions shouted across the room or across the I.M. screen. Try doing that on a J2EE project or a C++ library.

By the way, the same holds true for security. The bulk of my consulting income the last few years is doing security reviews, audits, penetration testing, and remediation from same. I go into huge shops at household name firms with excellent reputations and strong I.T. shops. The entire I.T. organization is managed and secured to strict standards. Except the four guys who manage “Notes”. They all have admin rights to everything, do what they need to do, what they want to do, and though they do have procedures there is no real enforceability on them outside the team. Why? The Notes stuff isn’t seen as a problem. It’s like the lights and the phones. Nobody gives it a second thought. It’s a huge system, but it’s managed by a small team of tech people. When you call them, they help you.

It is the very strength and stability of the core product that results in a lack of pain-based requirement to go with the kinds of change tools you see everywhere else. I’m not advocating ignoring the need for those tools, but I can certainly understand its derivation.

31 - @29 - I think part of the answer is that it was not clear the ITF toolkit allowed you to build good, scalable Domino apps much faster and in a consistent way." and it was clear that it tied you into some very specific ways of doing things, thus losing some of the flexibility of Notes/Domino. Notes/Domino is pretty good at RAD concepts all by itself, and there just was not enough incremental value.

The other part is closer to what you are getting at. If the demand isn't there, the supply doesn't matter much.

32 - The ITFactory architecture flopped not because it wasn't a tremendous engineering achievement, but rather because it WAS. It was simply not simple. As if to remind me of that fact, I recently found my THICK training manuals from the 5-day class I sat through back in the day (too late to help Nathan - sorry). The learning curve was pretty steep even if you could convince folks of the value of more disciplined development methodologies, which as several have observed is often an uphill battle. Getting the budgeteers to spring for an expensive tool that would require significant developer retraining (i.e. lost billable time) required a longer-term view than you typically find in US businesses. Couple that over-optimistic business plan with the IT crash of 2001 and the result was inevitable.

The lessons that I took away from this experience are central to my more recent endeavor to build and promote SuperNTF as a notes database framework that is easy to use and learn, but still include many "advanced" features. One of the other unique aspects of SuperNTF vis a vis other frameworks out there is that I am a big proponent of OPS (other peoples' stuff) methodology. If anyone reading this would like to contribute ideas or code to this effort, I'd love to talk to you Emoticon.

@30 - Just noticed Andrew's contribution, and it resonates a LOT with me. Very insightful. I came to Notes from the business side and "got it" almost immediately and have grown up with it ever since. I've never used a compiler and certainly understand why there would be tremendous resistance from "Notes people" to ever have to do so. That said I do have RDBMS experience and obviously break with many of my Notes brethren by giving a crap about disciplined development. I guess its just one more way that I find myself serving as a "bridge" between two very different worlds. SuperNTF is an outgrowth of this insofar as I'm trying to make discipline less scary for these types of folks.

Here are links to several open source Notes application frameworks that I have alluded to (these will all be covered to some degree in the "Templates, Templates" presentation Bruce and I are giving at Lotusphere):

SuperNTF: { Link }

OpenSlice: { Link }

Domino Application Framework (DAF): { Link }

(e)notes core: { Link }

33 - Nathan -- I definitely agree with Ben on this. "Good", "scalable" and "much faster" were far from obvious advantages of ITF. "More consistent" definitely was an obvious advantage, but not enough to get most people to buy.

But mainly, I think that mature Notes shops could see the value in ITF, but knew that achieving maximum value meant re-working lots of their old projects -- with not a lot of ROI to the owners of those apps, therefore no funding. And new Notes shops didn't see the value because they bought into the story from IBM that "all you need is Notes and Domino", which is in fact a true story... if you ignore such things as software engineering, lifecycle management, re-use, etc. And Andrew's response really gets to the heart of why Notes shops didn't think those parts of the story were important.

34 - I think we have much different ideas of what a packaged component is. Design elements that are inherited into every database that needs the functionality they provide is not a packaged component to me for one simple reason: they can be changed. So you create a subform and a script library and you inherit it across everything and all is good. Then someone comes along and changes the script library in one of the inherited copies, breaks the inheritance, and one app out of 50 gets hosed.

A framework shouldn't be held together by chewing gum and a prayer that nothing goes wrong. Yeah, you can beat on Notes enough and come up with something resembling a framework. Or you can decide it's not worth the effort and just continue custom coding everything. I do some amount of reuse, but the barrier to widespread use it simply too high.

I've never found any way to effectively componentize and modularize Lotus design elements the way you can with a OCX. If I could, I'd be all over that.

Don't get me started on LotusScript classes. I want to treat whoever came up with that abomination to some Serpent And The Rainbow style interrogation. Emoticon

35 - Hi, sorry to come a bit late to the party, but that's exactly what we have tried to accomplish with openslice <a>http:\\www.openslice.com</a> We are using this in the organisation I work with to great success. I know I was part of the development of openslice, but we have a team of 10 developers, all using it as the foundation for every app we have built in the last 2 years. New developers to the team have embraced it quickly without too many complaints. We also have quite a few other organisations in Sydney that I know are using it for the basis of new applications as well, and we have had lots of emails/comments from other areas that we think are using it.

Again, I know it's my own baby (with Tim Rynne), but I can't imagine working without it now.

I think a big part of the problem in Notes is definitely people just falling into a bit of a development "rut", and not wanting to seek out new, better ways of doing things. Once the effort is made to change, I don't think many would go back.

36 - @30 - I guess I should point out that I never felt the need to go over a history of Notes dev 'cause, honestly, I lived it. Other than a WordPerfect macro, the very first programming I ever did was a Notes filter & view at Relavis. And yeah, I lived through the days of fighting the "I prefer cc:Mail," "OS/2 isn't scalable," "Notes stores in a flat file," and my personal favorite "It's just email on steroids."

Andrew, I read all that and I take away the idea that Domino developers indeed have a culture that eschews formalization, right? We have that tradition because of where the product came from, and the fact that since we never HAD to have formalized processes before, we don't want to be REQUIRED to have them now. So we seem to be in close agreement on this matter.

@32 - We're going to need a lot of bridge building to get the Domino world thinking more like "real programmers."

@33 - I don't know what the price point on ITF was, and I see your point about cost of conversion. Nevertheless, given the state of the art of Domino development at the time, it seems like a no-brainer to me.

Perhaps what they missed was really good WEB tools (I only worked on a client app with it.) That was in the timeframe when everyone was jumping ship on the rich client.

@34 - Charles, allow me to introduce you to "Hide formulas and Lotuscript." You can lock down a script library that's cross-template inherited exactly like you'd lock down an OCX. You release object code only.

Like an OCX, you can just cut one version and distribute from there if you want. Or you can think of the Design task as being Domino's version of auto-update going all the way back to version 4. Any given developer who's adding the component can just to turn auto-update for it on or off.

I'm sorry your experience with LS custom classes hasn't been positive. Personally, I don't write anything else anymore.

37 - Your point about toolkits (and the lack of) was what I was trying to get at with this { Link } post on ideajam.com.

Now maybe I posted it in the wrong space or maybe my explanation was weak - but the response has been tepid!


38 - "how many developers have ever taken the time to learn about cross-template design inheritance? 1%?"

Sadly Nathan I think you may be optimistic in your estimate. Emoticon

(Let me say, that my following comments almost certainly don't apply to anyone reading as you are obviously pro-active in improving your development skills and practice.)

I reuse code wherever I can all the time. I have a standard set of over 20 templates to 'drop-in' specific functionality into new application-templates as-and-when I need them. If I improve/fix one of these generic-code templates (eg my XML-handling template) then I simply refresh all the application-template to roll this out to all my applications. Neat! Emoticon

Doing this is second-nature to me, but this is because I come from a big-iron RDBMS-based development background where a single application could have tens-of-thousands of design-elements with goodness knows how much code in each element. Before that I did a Computer Science degree where the focus was upon concepts and best-practice rather than whatever-was the in-vogue software of the time.

(Don't get me wrong, I do enjoy the RAD approach that Notes offers and much prefer doing test-driven development than the traditional approach of essentially doing all the coding and followed by a huge test and fix session at the end of development.)

However I frequently find that Notes development has been done by individuals who have only ever done small pc-based development, almost exclusively undertaking one-man projects.
Consequently they have never had to use the rigour/ disciplines required for large-team based projects. (Another benefit of such large environments is learning from the more experienced memebers of the team as to what is good-practice and what is not.)
Often, simply because Notes is so-easy to develop-in, the 'developers' turn-out to be Admins or enthusiastic end-users who have migrated themselves into developers.

Now some of the applications produced are novel and can-be insiteful from a business-rules perspective, but I also find they tend to be weak on technique/design.

Since moving to Notes developemnt 10 years ago I usually have one project 'on-the-go' that involves a complete rewrite of an existing application and/or data-correction work.
Common problems are;
(a) developers changing the 'data-schema' of an application but not doing anything to mould the existing data into the new schema, resulting in data variation simply-by when it was last edited/created
(b) scalability issues, can anyone top a database I found with over 300 views?!?
(c) poor and inconsistent appearance and user-interface (either reminiscent of a late 90's porn website or with dozens of animated gifs giving your eyes sensory overload)
(d) redundant and obsolete code / design-elements left in situ within the template (assuming they've used a template...)
(e) zero code resuse, with business-rules being recoded seperately in dozens of places (usually never 'in sync' with each other)
(f) ok, real developers hate doing this too, but zero documentation seems to be the norm

I could go on.... but poorly designed/buggy Notes applications help-to undermine both the standing of Notes Developers and the continued-use of Notes in companies.

39 - @Sean J,

Can you ping me off line? I have a favor to ask of you.

40 - Ah, okay. I've never hidden the design of a database other than as a test to see how it works. It seems to me that design hiding coupled with cross-template inheritance would get complicated in a hurry. When you make a change it requires updating the base NTF, refreshing the design of the second-level NTF, then deploying that to the server and refreshing the design of the databases.

I've never been a fan of cross-template since there is no UI to tell you which design elements inherit from what template. I suppose you could name the design elements in such a way that the name of the element tells you where it originated, but like the design hiding, it's a lot to go through. I just don't see the end result of either laborious process being worth the effort.

I have used LS custom classes and my problem is their implementation and the lack of integration. Like the ideas of cross-template design inheritance and hidden designs to create blackboxed components, I find the difficulty of implementation too high a barrier to widespread adoption. And I do blame that on the framework (despite you calling that BS). Developers have a right to demand that the platform be tooled to make them more productive rather than having to jump through purple flaming hoops just to achieve something that is easily done in every other modern development environment.

I probably fall into your 1% who takes the time to investigate some of these things. I just choose not to adopt them. Emoticon As I said before, if you beat on these things enough and you're willing to accept "mostly good enough -- sometimes", these concepts can be applied and made to work. I'm a little more pragmatic, set my expectations slightly lower, and go a more rudimentary route.

Maybe I'm deluded or an egomaniac, but I don't think that's a failing of me as a developer. The cultural phenomenon of Notes development being prototype heavy and adverse to reuse is because that's how the framework steers it. You can buck the system or you can go with the flow. Once the Notes development tools reach parity with the rest of the industry I'll embrace the techniques you advocate. Until then, for me it's just too damned difficult to be worthwhile.

41 - @38 - Domino development is all about RAD... that means that you can ignore all those points you made in a-f. Emoticon

I absolutely agree with your final point. Its a recurring comment on our site, even though the people who are making it don't even realize they are doing so.


42 - @40 - Charles, if you don't find it more productive to write OO code instead of procedural code, no amount of updates to Designer is going to change that. That's a design philosophy issue, not a tool issue.

I don't think you're deluded or an egomaniac, but honestly, I think what you just said was lazy. You outlined all the difficulties in managing the distribution and maintenance of closed-source inheritable design elements, but you compare it to OCXes!??!?! What the source-maintenance and deployment model like for THOSE!?

Copying & pasting a design element from NSF A to NSF B and clicking "yes" is NOT jumping through hoops, sir.

43 - @42 Nathan, there are an awful lot of Domino developers who have been saying FOR YEARS that Domino Designer's support for OO is so bad they don't use it. That's hardly news. Saying that everyone should be writing OO code anyway, when the tool not only discourages but actively punishes it is quite a stretch.

44 - @43 - "Actively punishes!?" Why? Because you have to actively segment your routines if you want to have them appear in the navigator?

You don't get type-ahead if you write procedural code, so that's no advantage. You can't derive from product classes if you write procedural code. You don't get optional parameters (method overloading) if you write procedural code.

So seriously, what is the pain DIFFERENCE between writing OO code and procedural code? The only thing I can think of is what's in the code navigator. That's a pretty trivial thing to drive people to write architecturally bad code.

Lotuscript might not be a good OO dev environment today, but guess what? It's not any better a procedural dev environment either. So you might as well write good code in the first place.

And when we get a proper code navigator in 8.5, guess who's existing deployments is going to be ready to leverage it?

45 - You can also use Script Browser { Link } and have some of that now! Emoticon

46 - My view is that the whole Domino architecture is simply too unstructured to ever lend itself to the sort of IDE/tools which other environments have.

I wrote Domilipse Java as a 'hello world' application to see what Eclipse was all about and to learn some Java.

Trying to get Eclipse to see a Domino application as an 'organic whole' was like trying to fit a square peg in a round hole.

There are too many things which are so simple to do in Domino but throw a real spanner in the works of any tool which tries to treat an nsf as anything other than a set of disconnected and unrelated design elements.

I have no idea what DDE will be like so I'm looking forward to seeing exactly what it will do and then eating my words. However, I wonder if the 'connectedness' which the JDT brings to Java development will ever be possible in DDE.

Let me give an example.

Agent A 'uses' Script Lib X. Make a 'backup' copy of Agent A and call it Agent B. Bad practice perhaps but something I see all over the place.

Modify Script Lib X and update Agent A to reflect changes in X.

We now have a prefectly good Agent A/Lib X but Agent B is junk and won't compile.

The current IDE has no problem with junk/uncompilable code in the database because it doesn't see the nsf as a whole.

If we import these elements into Eclipse loads of error will be flagged in Agent B which can't and shouldn't be 'fixed'.

Another problem is that both Agent A and B will have duplicate class names, which is a big no-no in Java projects. etc etc

$0.02

47 - @44 Well, if you don't consider the hoops Domino Designer makes you jump through to be a form of punishment, I don't know what to tell you. Your pain threshold is obviously higher than mine.

My main point, though, is this: building OO code has a learning curve, and it is (initially) significantly more difficult than building procedural code. Building OO code in Domino Designer doesn't have a learning curve, it has a learning *wall*. If you are already an accomplished OO developer, you can make Domino Designer work for you. If you are a neophyte, the pain points that you (rather cavalierly) dismissed become serious disincentives.

We ARE talking about why Domino developers don't follow more best practices like frameworks and OO and standard processes, right? Why is it not relevant that the only tool those developers have to work with makes it much more difficult to do those things than it is for developers in other environments?

It is my contention that the feedback cycle in Domino development is heavily negative with respect to OO, among other things. You seem to be saying that isn't an issue. Is that really your position? That Domino Designer itself isn't part of the problem, but it's ALL due to the fact that Domino developers are lazy and ignorant? Really?

48 - I use OOP daily in Access and VB because it's well-integrated and easily accessible. Your argument that LS OO classes suck no more than LS procedural code is pretty damned funny as part of a conversation about why Notes developers don't use best practices. Emoticon

I don't use OOP in Notes because it's poorly integrated, difficult to access, and the UI for it is a mess. Procedural code works no better, but it's slightly more convenient. If I have to choose between annoying and moderately more useful, or less annoying and still functional, I'll go the latter route. Maybe that makes me lazy. As it stands I don't see the pain of LS OOP development to be worth the effort.

49 - @47 & 48 - Gentlemen...

Please outline for me the "pain points" of doing OOP in Lotuscript that are specific to OO. That is, leave out the issues that also exist in procedural programming, whether because of the IDE (no typeahead on custom code) or because procedural doesn't support them in the first place (can't derive from product classes.)

Feel free to conspire on your list. Consider this a promise to take said list directly to Maureen in the next AppDev DP group discussion. But I think this is a red herring.

50 - @48 - "Your argument that LS OO classes suck no more than LS procedural code is pretty damned funny as part of a conversation about why Notes developers don't use best practices."

I've read that 4 times, and it makes no sense to me. If it sucks to write OO code, and it sucks to write procedural code, WHY WOULD YOU WRITE PROCEDURAL CODE!?!

If you don't like the IDE, you don't like the IDE. But one coding style is not worse than the other in the IDE. It's just an old IDE, period.

51 - @49 The lack of an 'Interface' construct is one thing which is missing.

52 - @51 - Is there an 'Interface' construct in procedural programming? I'm pretty sure there is no such thing. Which means the lack of it is not a reason to write procedural code instead.

53 - @49 So it is your contention that Domino developers are to blame for the problem, and the IDE itself is not an issue. Wow. I hope you're playing Devil's Advocate, Nathan, because your claim that OO is not more difficult than procedural code in Domino Designer is just bizarre.

Following that logic, why should IBM bother giving us DDE? We have everything we need already, right? We should all just stop whining. On the plus side, there's no reason for anyone to waste their time learning the Eclipse IDE, because we just don't need it. Such a shame IBM is going to force us to waste all that effort just to write code in 8.5, when we could happily be using Domino Designer. /snark

You're citing things that are issues in OO (no type-ahead, no class browser) and claiming that they don't count because the same problem exist for procedural code? News flash: we don't NEED those things to write high quality procedural code. And procedural development environments don't support those items as a rule (wait, ARE there any purely procedural development environments left? other than Domino Designer, that is?). OO environments do.

54 - @53 - So Rob, your contention is that you NEED type-ahead to write OO code, but not to write procedural code?

Bullshit.

55 - @52 You are right that there is no Interface construct in procedural programming.

That's because the concept doesn't and cannot exist in procedural programming as far as I can tell.

The lack of Interfaces is a 'pain' for those who write OO code.

On the other hand I'm probably missing the nuances of the question you're asking. Emoticon

56 - @54 Call BS on me all you like. I'll call you *deliberately* obtuse.

OO code is inherently more complex and in that sort of environment type-ahead and a freaking class browser go from being handy little tools to absolute necessities. Yes. Damn straight. You NEED them to write OO code and you don't NEED them to write procedurally.

If you're wasting the time to make your OO code no more complex than your procedural code would have been, all you're doing is putting lipstick on the pig. The fact that you can make the pig moderately more attractive (actually, not sure of that - is lipstick on a pig actually an improvement?) does not make it any less a pig.

57 - @53 Procedural programs have a very, if not entirely, flat hierarchy.

In a truly OO environment, such as Eclipse, a single class might implement multiple Interfaces and extend a class which extends another class which itself might implement multiple interfaces and so on.

Without type-ahead you can spend ages just trying to figure out what methods a class has.

In procedural programming the methods you see are the methods you get so the problem never arises.

58 - @55 - Keith, my contention is that the list of things that are painful in OO that aren't ALSO painful in procedural programming is really, really short.

@56 - Strange. I've done nothing but OO development in Lotuscript since 1999. Since I NEED type-ahead and a class browser to do that, and I don't HAVE them -- I guess I'm living in some sort of alternate reality. Or maybe you can just call me Arthur Denton { Link }

59 - @57 - So you mean, I need to either be able to read source code (so I can identify members of a class) or have documentation for OO stuff.

But in procedural, I either need to be able to read source code (so I can identify functions and sub and their parameters) or have documentation.

The only difference is that with procedural code, I can see the list of function & sub names (but not parameters) in the code navigator. Big whoop.

Flat vs. hierarchical is another red herring. I've seen plenty of procedural code that calls subroutines in libraries that nest 8 or 9 levels deep.

60 - @58 One could argue that OO programming in LS is really just OO-lite because of a catch-22 situation.

OO in LS is relatively simplistic because the tools/language don't facilitate advanced OO therefore the tools aren't needed because LS is only used for OO-lite. Kinda.

We could just call you Arthur Dent, blow your planet up and build a by-pass instead. Emoticon

61 - @60 - I agree wholeheartedly that we need more OO capabilities in LS, both from an IDE and a language standpoint.

What's ridiculous to me is the contention that those needs are a reason to continue writing procedural code.

62 - @59 Having a procedure which *calls* lots of other stuff is a pain if you want to *understand* the procedure but not really a problem if you want to *call* the procedure. Of course one still needs to know *what* procedures are available but the flat hierarchy still means that the methods you see are the methods you get.

On the other hand if a class *extends* 8 other classes *calling* a method can *only* be done if you analyse all 8 classes to see *what* methods are available.

The problems might seem the same but I feel that *in practice* not having type-ahead / Hierarchy view in an OO environment like Eclipse would make it almost impossible to program.

@61 I don't agree with that contention either. However, the lack of capabilities are a *disincentive* to write OO, especially for more junior programmers, so people carry on writing procedural code.

63 - @58 "Strange. I've done nothing but OO development in Lotuscript since 1999. Since I NEED type-ahead and a class browser to do that, and I don't HAVE them -- I guess I'm living in some sort of alternate reality."

No, Nathan, it just means that you are like unto a God to the rest of the benighted, pathetic, lazy, and astonishingly stupid LotusScript developers. Emoticon

64 - I also have to disagree on the the type ahead and the navigation being required tools. It's certainly more convenient, and I wouldn't want to give it up, but I was writing OO code long before type-ahead or fancy code navigation existed and I made it through just fine..... I think I'm fine anyway....Emoticon

I have to say that the IDE has a huge role to play in making it easy for people to get their work done. If this thread has exposed anything it's that by in large, your average Notes developer is not going to go out of their way to do something that is A) hard to understand ( and OO is definitely in this camp - especially for a non programmer) and B) Constantly being beat up by the very tool that implied it wanted to help. For me, the OO aspects of LotusScript and the IDE we have to use it in are just another 50% feature from IBM. And I think we covered that already...Emoticon

65 - @62 - Let's be clear: I'm absolutely in love with the editor capabilities of Eclipse. There's not a snowball's chance in hell that I could code a Notes 8 plug-in without all the nifty type-ahead and browsing stuff that it does. (Although old habits die hard... I never look at the class browser for my classes. I keep forgetting that I CAN!)

But "almost impossible?" I just don't see where you guys are getting that kind of blanket statement. Have you TRIED it? And I mean for more than, like, 10 minutes.

As far as procedural vs. OO, I suspect that you are thinking of comparing a CLASS to a FUNCTION. That is not the unit of comparison. You compare a CLASS to an entire LIBRARY.

Yes, you can call individual functions without knowing ALL the functions available to you -- if you already know which one you need to call. Otherwise, you have to go figure it out. Which is EXACTLY what you do with you're dealing with a class.

I don't need to know EVERY method of a class to call ONE method of a class, any more than I need to know every function in a library in order to call one of them.

@63 - Hey, I'm only applying one of your four adjectives there. Emoticon

Rob, would it help if I sent some sample class libraries to you? They really aren't hard to follow. And they're *way* easier than the insane sequence of parameters dropped into subs that I see LS coders do all the time.

66 - @64 - See, I do NOT buy that OO is hard to understand for an LS programmer. If you write procedural code, you already HAVE to write OO code. You have to understand properties and methods just to work with NotesDatabases, NotesViews and NotesDocuments. What is the big mental leap from there to going "I think I'll have a ContactDocument and a ContactView that I define my own methods for!"

Every single Lotuscript developer on the planet already writes OO code. They just don't write their OWN OO code.

That, friends, is a cultural issue.

67 - @65 Thanks, but I've seen classes before, Nathan. I even managed to understand one. Once. But that was back in the days when I could still remember how to count higher than 8, so it was obviously an unusually productive time for me...

68 - @65 Have I ever TRIED Eclipse?

Pay attention Nathan!

Regarding my last plug-in someone said...

"Friggin awesome. Thanks so much! Nathan T. Freeman." Emoticon

69 - Mr. Frunobulax, I know who you are. Emoticon I meant, have you ever tried writing custom classes in Designer? Today, under current conditions.

I know you've done everything under the sun in Eclipse. Didn't mean to suggest otherwise.

70 - @69 I did wonder... Emoticon

My day job is writing bog-standard Notes 6.5 stuff so I write a lot of custom classes in LS... and 'yes', it's a pain.

I don't use Java, Eclipse or Notes 8 at all, other than as a 'hobby'.

I get the feeling this conversation is going round in circles. We all agree that doing OO in LS in IDE is LTI (Less Than Ideal).

Roll on DDE and then we can moan about something new!

71 - @66 - OK, so 'hard' may have been a bit much, but it is different enough to seem that way. And I do think that *using* a class is very different from creating your own. Just making use of the backend classes is not doing anything more than using an API with a funny syntax.
To really get value from OO you need to understand inheritance, encapsulation, and method overriding. Those are not trivial concepts for anyone to understand.

Thinking in an OO way is very different from a procedural way. Otherwise, as we say in C, all you have is structures with methods. Or in, in Notes terms, Types with Subs.

72 -
How sad that a good discussion degenerates into mindless name calling and argument of who's lazy and who's not.

Its fair to see I know a good bit about object oriented programming in lotuscript, visual basic, vb.net, c#, c++, java, and some languages I'm quite sure are too obscure to name. I have, on the same day, written in C & C++ for linux (customizing Asterisk), bash shell scripts, lotuscript, java, vb.net, javascript, html, & css.

I think I'm qualified to say that Lotuscript, at present, doesn't lend itself easily to putting everything into classes. Right now, I create custom classes (hell, in any other languages that terms is meaningless) for only a few of the more long term,higher complexity, or intentionally reusable projects I'm working on. Even within those projects, I reserve this kind of thing for specific complex tasks like rules engines or whatever.

Now, in vb.NET one of the first things I do I create classes outside the framework they give you that's so tied to the UI. I make full use of my own namespaces and object model. So what's missing in Lotuscript?

#1 - Obviously a class browser

#2 - You can't return arrays, lists, and custom objects easily. You can do it, but its messy and ugly.

#3 - its not a good language for dealing with complex data you're going to work with for a long time. There are no vectors, iterators, hashtables, or pointers built in.

#4 - Polymorphism? Overloading? Hello?

#5 - Threading? Multithreading? Semaphores?

Lotuscript is still a scripting language, not a programming language. Its good, quick, effective, and easy to work -- hell I can almost think in lotuscript -- but its limited.

So, am I lazy because I use Lotuscript mostly as a scripting language? Not many people get to call me lazy.

There are times for making objects. When you are writing something meant to be reused, or even meant to be used by other people. There are also times when going to the trouble of putting everything you do in a custom class is overkill in the same way as speaking between your code modules in xml just so you can say you're standards based is overkill.

Domino developers don't NEED the safety net provided by code management tools the way other developers do because the platform is forgiving. Its no that the tools aren't good, its that there isn't enough pain to justify the effort for most developers and the cost to the shops that employ them.

73 - In my self-inflicted role as "bridge builder", I would like to recommend that those LotusScripters (or is it LotusScribes?) still intimidated by the notion of OO and custom LS classes read an *excellent* series of articles by John P.M. Dillon that appeared earlier this year in Lotus Advisor. They will require a subscription to read and include sample code. The first article is here { Link } but do be sure to read all the related ones.

74 - I can't speak for any of the other people who have complained about Domino Designer and LotusScript's support for OO, but I don't personally have "issues" with OO design. It's not intimidating, it's not difficult, and it's not scary. I've done it in Java, I've done it in VB, and I've done it in JavaScript (yes, JavaScript, in 2000). In those cases I had an IDE and a language that allowed me to do real work. With exceptions, I just don't consider it to be worth doing with a toolset the equivalent of a crayon.

Andrew has graciously done what I stubbornly refused to do earlier, which is answer the question about what Designer/LotusScript is missing. I couldn't believe Nathan was serious, though I guess I have to admit that even for Nathan this would be taking a practical joke a bit far. Not that I'd put it past him, mind. This is not a dare, Nathan. Emoticon

75 - @72 - So Andrew, with all those myriad of languages that you're out there coding stuff in, do you use procedural Lotuscript as your primary data and UI processing tool? And if so, do you do it BECAUSE the OO capabilities of LS lack overloading, polymorphism and a built-in class browser?

"There are times for making objects. When you are writing something meant to be reused, or even meant to be used by other people." -- if you want to write stuff that's not for reuse or sharing, that's your choice. The same goes for Rob, or Charles, or anyone else who stumbles across this corner of the internet. We're all free to obsolete our code for the sake of our own expedience. Hell, I still write @formula agents when that's the fastest way to dash off a quick data sync agent.

It's just crap to blame these things on IBM. Sure, they've ignored the tool for half-a-decade, and that sucks. But nobody's development habits are going to change because they get a class browser. They're going to change because of an active decision to take reusability more seriously, and stop treating our OWN applications like toys.

It's like saying "well, I'd go to the gym if they had recombinant bikes and freeweights. But they only have treadmills and stationary bikes, so I just can't get the exercise I want. I think I'll have another Mallomar."

Andrew, you often write about your inventive and handy efforts around your home. You know why kind of workman blames his tools.

76 - Careful with the implications, Nathan. I'll ignore the overtone and try to answer the specifics.

I use lotuscript over java in the Notes client because:

#1 - The Java IDE in Designer is worse than the lotuscript one, and dropping into eclipse and back all the time is more work than I want to do for most of the quick tasks.

#2 - Java on the workstation takes time to load. It annoys me.

#3 - I know lotuscript like I know the inside of my eyelids. For what it does, it is effortless.

Now, as to your comment about reusable and not reuseable code:

Not all scripts will be reused. Not all code will be reused. I have my function libraries, and I have code that's been in production for more than ten years. That's not the majority of code. It also isn't necessary to create a custom class to have reuseable code. Well commented functions, efficient structure, tight segments of functionality -- all make code easily reused.

Any object is useful only if you're going to re-use the same object, against the same kind of structure. That's often not the case. As I said, for rules engines, for providing an abstraction layer to your more complex code. It just isn't always necessary or even helpful.

Its about what you're doing, Nathan. Hell, I have a three PRODUCTS around which the primary value is delivered by an exposed object model.

There's a time and a place. Lotuscript is great for what its great for, but if you're really building something big, it wouldn't be the modeling language of my choice.

77 - In my self-inflicted role as "bridge builder", I would like to recommend that those LotusScripters (or is it LotusScribes?) still intimidated by the notion of OO and custom LS classes read an *excellent* series of articles by John P.M. Dillon that appeared earlier this year in Lotus Advisor. They will require a subscription to read and include sample code. The first article is here { Link } but do be sure to read all the related ones.

78 - @76 - So Andrew, I didn't see the line in there that said "I'd write custom classes in Lotuscript if I had a class browser and could create members with optional parameters via method overloading, but since I don't have those features, I write procedurally."

I see stuff that says "I write procedural code in a reusable fashion" and I see "I don't do class modeling in Lotuscript (or in Java, at least for Notes.)"

Both are great answers. Thanks.

And there are no "overtones," Andrew. I buy that there are reasons to write procedural code over OO code. You give an excellent one -- because it's simpler to formulate in your mind and it doesn't matter whether you blackbox the result (or you can blackbox it well enough with a function.)

What I don't buy is that the lack of a class browser is a good reason to write procedural code where you know object code would serve you better. That is the argument I hear Domino developers make all the time. That's the argument being made in this thread.

79 - There's no way I can predict what I'd do with an enhanced Lotuscript that does the things I called lackings. Some are likely to be fixed soon -- as you well know. Others are so far away from even conceptions of being fixed that the sun will run out of fuel and go nova before it happens.

A language has syntax, features, and constructs. Then it has a zen. To really go geeky, calling "groking" a language. Lotuscript doesn't --in present form -- WANT to be an object modeling tool. It can be made to do it, but that's not how it wants to be used.

Give it real more powerful function declarations, a good class browser, type-ahead that works with my own objects -- well, who knows. For now, its still prone to memory leaks any time you call outside it memory model (to com/active-x, to lsx's, to command shells -- but never predictable, and it comes and goes in different versions).

The best hope I ever saw for Lotuscript was ESB (Enterprise Solution Builder) which was like a lotuscript servlet engine. Sadly, it never quite got there in terms of scale and failover tools that you need with something like that. It was big in some Japanese sites for a while, but it finally went away.

For what its worth, Java was the hardest "Zen" to learn of any language I've ever tried, because its the one where that's the most important. If you fight Java, you get nothing but "Class not found"

80 - @79 - I guess I just don't have the same relationship with code that you do, man. I'm not worried about a mystical sense of what my code wants. I'm just concerned about what *I* want.

81 - @80. Really?

You don't every get the feeling that sometimes you are really fighting the code and sometimes it's just writing itself? That sometimes everything feels right and at other times it's just oh so wrong?

Interesting.

--
Kerr

82 - You all make me glad I spend most of my time writing in C with a smattering of C++. In TextPad.

83 - Ben, for C with a smattering of C++, that means I'm writing for Asterisk on linux and that means I'm in over an ssh shell using JOE (Joe's Own Editor) -- essentially a text editor with highlighting and wordstar command keys. Emoticon

84 - For a couple of posts before, as this is late coming...
It's not that IBM does not listen to it's customers.., it's that it cannot.
That's right, it CANNOT listen to its customers.
There is no mechanism for internal connectivity between the existing silos, nevermind the farm. They can use all the Notes, connections, quickplaces, sametimes and portals they wish, but they are missing one key component.
The reason why companies like lotus 911 and PSC are successful at what they do is because the contain "routers"...
What good is a domino server without its router? it can collect a whole lot of information, manipulate it,present it, delete it or archive it, but without the router, everything remains static.
Hell, just think of the mail.box, why do messages just sit there for no reason? why do they release or not, route or not, go dead or not? (no one get smart on me) sure sounds like IBM
But my point should be clear.
IBM is Domino with the router task hung.
If i can only get them to let us Load Router....Ill try yet again at LS'08

85 - @83 - It beats using vi.

86 - For what it's worth here is my $0.02

I think Andrew has steered the discussion to where it is apparent what the essence of the issue is. LotusScript as a language often feels like it needs too much coercion where the opposite should be happening. It should be getting out of my way.

Partly it's a case of providing just enough features to appear fully developed and capable but not enough to actually be that. As an aside, I often find that the rest of the framework has similar issues. Just look at what hoops we have to jump through to accomplish what others take for granted. Take the Revolution. It is neither intuitive nor an ideal solution. What it provides and more should be part of the vanilla product since scripting was added. Yes other systems cannot do what Domino does but this is no reason to not provide basics that others take for granted that in now way conflict with the Heart of Domino.

87 - Why don't Domino devs have a culture of reusability?
That sounds naïve, if I can be less than delicate but direct. Sorry. Reuse is a 20+ year old dream (or is it closer to 30 now?). Just because some shop does OO it does not mean that they do reuse. Just because they use inheritance does not mean they reuse. Just because they package code into class structures doe snot mean they do reuse. And Domino development is no special case that would somehow buck that trend. It does not provide any revolutionary feature not available elsewhere in the industry that make reuse an ingrained practice.

In the real world reuse happens mostly at the level of frameworks or APIs. It rarely happens at the higher application specific layer unless there is a concerted, continual and concentrated effort towards that end. There has to be a culture of reuse regardless of technology. And just because you give people a component based technology (or any other made with reuse as a design goal) it does not follow nor guarantee that reuse will happen. As an industry we have continually missed the mark on reuse. The barriers unfortunately far outweigh the benefits.

Having said that I do agree that it is an admirable goal to set. It is a long term goal to be sure that over long term can pay handsome dividends. Are we prepared to be educated about reuse and to make other short term investments? And are we ready for a world were reuse is as much a part of our work as anything? It might be a world where Domino teams become even smaller due to increased effectiveness derived from reuse.

As for Domino teams doing it right I think that when the developers are ex-power users or administrators it is less likely but when you have developers come into Domino development from some other world (usually bigger and more structured) they often bring with them and insist on the practices that they have used elsewhere. And sometimes they bemoan the lack of tool support to fully implement such practices such as source control, automatic builds, language support, etc

One last point. I think we (mostly) agree on Domino being perceived less than it is, domino developers being an afterthought in the IT budget, etc. So what are we going to do about it?!

88 - Nathan and others have identified the meagre amount of OO code produced and available for Lotus Notes applications.

I'd like to see a few people write about how they create their OO code in their blogs. Once developers see how to create the code and how it can be used, they will use it.

89 - At Lotusphere there will be a regular session "Object Oriented Programming with LotusScript – Take It To the Next Level" { Link } and a BoF session "Object-Oriented Programming (OOP) and Frameworks in LotusScript" { Link } .
I guess its a beginning.

(Yes, I know. Both session are damned early i