Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

There is a plethora of different programs, servers, and in general technologies in use in organizations today. We, programmers, have lots of different tools at our disposal to help solve various different data, and communication challenges in an organization. Does anyone know if anyone has done an serious thinking about how systems are integrated?

Let me give an example: Hypothetically, let's say I own a company that makes specialized suits a'la Iron Man. In the area of production, I have CAD tools, machining tools, payroll, project management, and asset management tools to name a few. I also have nice design space, where designers show off their designs on big displays, some touch, some traditional. Oh, and I also have one of these new fangled LEED Platinum buildings and it has number of different computer controlled systems, like smart window shutters that close when people are in the room, a HVAC system that adjusts depending on the number of people in the building, etc.

What I want to know is if anyone has done any scientific work on trying to figure out how to hook all these pieces together, so that say my access control system is hooked to my payroll system, and my phone system allowing my never to swipe a time card, and to have my phone follow me throughout the building. This problem is also more than a technology challenge. Every technology implementation enables certain human behaviours, so the human must also be considered as a part of the system.

Has anyone done any work in how effectively weave these components together?

FYI: I am not trying to build a system. I want to know if anyone has thoroughly studied the process of doing a large integration project, how they develop their requirements, how they studied the human behaviors, etc.

share|improve this question

closed as too broad by gnat, GlenH7, MichaelT, Kilian Foth, jwenting Sep 29 at 11:01

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

    
    
I later found this link, but left the question open to see if the dialog would continue. I will leave it open a couple more days and then close it. Are these theories used in industry? –  Jeffrey Guenther Jun 27 '11 at 15:57
    
    
eaipatterns.com –  Bossliaw Jul 20 '13 at 19:46

6 Answers 6

I don't know if there is any theory, but here is what I've experienced working with small and medium size businesses (Which is where all businesses start.).

  • limited to no IT expertise is utilized early on. Integration is a glimmer in a future CIO's eye and a CFO's nightmare.
  • small applications are purchased as needed to solve specific problems: email, website, contacts, proposals, document creation, accounting and usually something specific to some niche industry.
  • off-the-shelf small scale applications are notorious for making it easy to query/report on the database, but terrible at bringing data in (unless you type it). Rarely have an API.
  • willingness to duplicate efforts/enter same info. in different systems.
  • reporting is done ad hoc since they're still learning.
  • more willing to hire a new employee for 25K/year than buy 20K worth of software to do the same job.

Eventually the business grows to the point where this becomes a mess. Wouldn't it be great to create one application that could do all this? Too many specific tasks that all these different apps do better than what could be rebuilt within a reasonable amount of time and money. No one wants to give up any of the existing functionality. All of the business processes have been poorly documented. No one knows why the Assistant to the Assistant prints out the XYZ report every month and puts it in a folder, so obviously it can't be eliminated.

No one wants to build applications that hold unlimited reference keys to the same data in other systems or flexible import capabilities. That would be over-engineering/pre-optimization/un Feng Shui/detriment to the sublimely simplistic beautiful code. And all the clients bought the software without it so why add it?

Maybe we should just put it all in xml?

share|improve this answer
    
You hit the nail on the head. This is exactly the problem I talking about. It seems like design/engineering have a lot to offer, but are dependent on a clearly defined problem? What about systems, like the ones you described, that aren't clearly defined and evolve over time? How can systems be designed to support evolution? –  Jeffrey Guenther Jun 29 '11 at 15:44

Interop is one way how big companies make money. Oh - if you use Windows, you would want Office as well. You might also need VS to develop stuff for windows. How about Internet explorer - and so on. Ecosystems are bread and butter for building huge companies.

Having said that, there is no one way to integrate disparate systems. But people are working towards it all the time. Here are a few:

  1. REST based services
  2. Internal platforms to build your websites/services on. These platforms can take care of the hard stuff like Synchronous / Asynchronous messaging, monitoring, load balancing, fault tolerance and so on while the LOB developers can handle the business part alone.
  3. System level interop technologies like COM/OLE (I know I am rusty here:()

This list goes on and on. One way most of the companies evolved have been by getting a monolithic setup up and running, then factoring out pieces as the company grows. The new startups have it easy in a way that many of these platform / infrastructure pieces are out there. Keep an eye out for the big picture is all the answer there is.

share|improve this answer
    
What about the people aspect of the equation? It is fine to have the bits moving the right direction, but what about organizational structures and their influence on the tech.? –  Jeffrey Guenther May 28 '11 at 3:33
    
The age old way of communication! We usually have architects who span projects - these folks will define the interfaces etc. at a broad level and review with the actual teams to improve on it. Once that is solid, there may be a smoke screen system that goes up, but usually people work from the interfaces towards their actual systems. –  Subu Subramanian May 28 '11 at 3:36
    
You raise the point of big companies creating their own ecology of applications. I can see why the would do that. It makes it easy for someone to slap down their briefcase of cash and get a system that plays well with itself. What if the applications cross domains and you are dealing with a heterogenous mix of "best tool for the job" technologies? What about application that are not networkable like AutoCad or Photoshop. –  Jeffrey Guenther May 28 '11 at 3:46
    
You have to find integration solutions then. if you are working off a windows stack, you could use Sharepoint servers to share your documents from autocad/photoshop etc. –  Subu Subramanian May 28 '11 at 3:48
    
The challenge of finding those integrations solutions is exactly my point. From a user experience point of view, how well the system are integrated and how they are integrated are a big deal. Are there research methodologies? Design patterns? –  Jeffrey Guenther May 28 '11 at 3:57

This is the problem that ESBs and integration patterns are trying to answer. Event-driven distributed architectures are one way to go, but they're still pretty new. ISTR that Bell Labs had a system that tracked employees as they moved around the building, and would forward their calls to wherever they happened to be, but that was just a research project, I haven't heard of anyone doing anything similar recently.

share|improve this answer

Back in the 90s XML was going to be the silver bullet which would solve all these problems (or at least be a key part of a solution). The work continues under the umbrella of OASIS (Organization for the Advancement of Structured Information Standards).

share|improve this answer

Yes this integration protocol is called HTTP and the Perl language is very popular for implementation of these integration systems!

share|improve this answer

I found "Software Fundamentals" by David Parnas talked about systems integration a lot. It's a book collecting some of his best papers over the last several decades.

http://www.amazon.com/Software-Fundamentals-Collected-Papers-Parnas/dp/0201703696

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.