July 20, 2013
The refactoring of everything
I’ll start with three observations:
- Computer systems can’t be entirely tightly coupled — nothing would ever get developed or tested.
- Computer systems can’t be entirely loosely coupled — nothing would ever get optimized, in performance and functionality alike.
- In an ongoing trend, there is and will be dramatic refactoring as to which connections wind up being loose or tight.
As written, that’s probably pretty obvious. Even so, it’s easy to forget just how pervasive the refactoring is and is likely to be. Let’s survey some examples first, and then speculate about consequences.
- NoSQL and schema-on-read.
- The main point of relational DBMS is the Ted Codd guarantee, which says that applications and database designs can be loosely coupled. The price is that database designs for different applications are tightly coupled into one comprehensive schema.
- NoSQL and dynamic schemas turn that around. For any one application, application design and database design are tightly coupled; but database designs for different apps are often unrelated.
- What you think of these alternatives probably has a lot to do with what you think about separating the developer and DBA job functions. If you like that separation, the relational approach should look good; if you don’t, then dynamic schemas may be more suitable.
- BI with dedicated data stores. Instead of just running against relational DBMS, various business intelligence tools feature proprietary data stores, often memory-centric. Two examples I’ve written about are Platfora and QlikView, but “in-memory BI” goes far beyond those two vendors.
- BI integrated into operational apps. A trend that’s been developing for years is the tight integration of BI into operational apps. I’ve written mainly about Workday’s version of this, but it’s at least as big an issue going forward in competition among SAP, Oracle, and most other application vendors you can think of.
- SAP HANA and competitors. It’s an understatement to call SAP HANA “overhyped”. But technology will surely someday get to the point that SAP implies is already here, with a lot of the silo-merging that that suggests. For databases that grow at much less than Moore’s Law speeds, it will be possible to integrate in-memory database capabilities that previously called for a variety of disk-based specialty systems.
- Analytic application subsystems. Customer-facing analytic applications will have a whole different standard of completeness than more traditional back-office transactional ones. The base case is analytic subsystems loosely coupled to a variety of front-end technologies.
- Other “real-time analytics”. I expected the short-request/analytic distinction to blur, but even so I’m astonished by the number of NoSQL and NewSQL vendors who’ve adopted “real-time analytics” as a core message. (For more on that, I refer you again to my recent webinar on the topic.)
- Appliances. For a variety of technical and business reasons, vendors love selling appliances, aka “engineered systems”. Frankly, some of the integrations between hardware, operating system, and other software are tighter than others, so only some of this is a true refactoring. Anyhow, appliance stories can be heard from a large fraction of the computer industry, including for example:
- Apple and other mobile device makers.
- IBM — PureThisAndThat.
- Oracle — ExaEverything.
- Teradata.
- Microsoft — Xbox, Surface, Parallel Data Warehouse, etc.
- SAP — HANA appliances
- The whole telecom equipment industry.
- Cluster management. We have entered the era of cluster computing. This has several consequences:
- Software designed to run on dedicated clusters often has cluster management software integrated in.
- Virtualization has evolved to break the old pairing between applications and the specific servers on which they run.
- OpenStack and similar cloud stacks are trying to take the evolution further.
- SaaS/IaaS/DBaaS/PaaS. Software as a Service, in all its acronymic variations, can integrate software, hardware, the surrounding bricks and mortar, service, and everything else. Conversely, different SaaS systems can be a lot more stand-offish from each other than multiple vendors’ packaged apps, all running in the same data center, perhaps even on the same DBMS.
I could keep going on for a while; for example, I haven’t said anything yet about “intelligent storage”; indeed, I haven’t even mentioned analytic platforms or their SQL-on-Hadoop cousins. But hopefully I’ve run through enough different cases to justify the slightly hyperbolic title of this post.
So what are some possible implications? My candidates start:
- As previously noted, I expect most computing to eventually wind up on a combination of appliances, clusters and/or clouds. That applies even to organizations whose workloads are small enough to run on single servers, because most of their computing (except for personal devices) will either be SaaS, or else the kind of public-facing internet applications that already tend to be in the cloud.
- Also as previously noted, I expect traditional databases — i.e. ones that focus on human-generated data — to eventually wind up in RAM. I imagine that there will be both relational and dynamic-schema APIs to the memory-centric DBMS that manage them.
- The current and near-future technology stacks underneath application suites such as Oracle’s, SAP’s or Infor’s are of little importance to their ultimate success. (Yes, that applies even to the wonder of HANA.) Much more significant will be the subsequent cloud/SaaS generation.
- Similarly, I think there is plenty of opportunity for large new application software companies, SaaS or not as the case may be, just as there usually is in connection with major technological change.
- Notwithstanding the various points of integration between analytics and short-request processing, there will also be analytics-only technology stacks for a long time to come.
The IT industry seems likely to remain interesting for a long time to come.
Categories: Business intelligence, Cloud computing, Clustering, Data models and architecture, Exadata, IBM and DB2, In-memory DBMS, Memory-centric data management, Microsoft and SQL*Server, NewSQL, NoSQL, Oracle, SAP AG, Software as a Service (SaaS), Telecommunications, Teradata, Workday
Subscribe to our complete feed!
Comments
5 Responses to “The refactoring of everything”
Leave a Reply
[…] Everything will end up on Appliances, Clusters or Cloud. The refactoring of everything […]
[…] and graphs integrated or at least well-embedded into production applications. This technology is on a long-term rise. But in some sense, integrated reporting has been around since the invention of […]
[…] The refactoring of everything (July, 2013) […]
[…] was perhaps my previous most comprehensive post on innovation, I drew a link between innovation and refactoring, where what was being refactored was “everything”. Even so, I’ve been ignoring a […]
[…] you factor things is always […]