SOFTware is HARD

I started using JUDE in the early days of UML modelling and have continued using Astah until now. In the day and age of Agile development, emphasizing "communication over documentation", is UML (and related UML tools) still relevant? I'll give a resounding YES! to the question. While we should do away with extensive documentation , we should still be able to communicate our intent in forms of architecture, design, workflow, processes, etc. In the course of my work in an end-user environment, I use Astah extensively to communicate with development teams, users, vendors and even senior management.  I find it's often a lot easier to use a tool like Astah to draw UML (or pseudo-UML) diagrams than pen-and-paper (or even napkins!). For instance, I can easily move things around, redraw and append/ update the diagram using the tool but not with paper. Some examples of diagrams I'd used in the course of work are below. I used Astah for the following: Share my thought proc...

UML appears to be dying in the light of the Agile development movement. working software over comprehensive documentation Many have wrongly interpreted this to mean no documentation. I recently read up C4 model for software architecture that describes the software architecture of a system from different depth & perspectives and for a different audience. The author promotes the use of a common vocabulary to describe software architecture using simple-to-understand diagrams. I attempt to do similar in this post using the same case study as that of the C4 article. The differences are: I do not wish to go into  Class (the 4th C) as that requires too much implementation details I replace the 4th C with Concept as I personally find the Conceptual model more relevant and important in appreciating the business domain.   Using Astah , I figured out how I can model the 4C software architecture efficiently using UML. Mind-Mapping I started with a Mind...

I was once involved in reviewing our company’s application architecture to identify security risks and recommend possible mitigation solutions. Since every organisation has its own unique constraints, available resources, and risk appetite, the solution herein might not apply to other organisations. Nevertheless, this post showcases what one can do with UML to document and explain (pictorially) security flaws in application architecture. The as-is architecture follows. A detailed explanation follows: The system (under design) is a Java Enterprise Application that is deployed in AppSvr (Application Server) within the Internal zone. The deployed application consists of both HTML-based as well as SOAP-based services. The 2 WebSvr’s ([Apache] Web Server) in the DMZ act as proxies. There is a load-balancing appliance used to balance requests to the web servers. A request pool has been created on the appliance (WebPool). The (public) user would access the URL endpoint of the appli...

The Enterprise Architecture comprises the following IT dimensions: Program Management Software Architecture Data Architecture Network Architecture Platform Architecture Business Architecture (which over-arches all the above 5) Of interest, are the roles of the Solution Architect and the Business Architect. Solution Architect The Solution Architect needs to be proficient in all 5 dimensions (with the exception of the Business Architecture). Being proficient is defined as being able to (at least): set standards, directions, define best practices lead large projects and develop programme plans establish policies, designs and directions As such, the Solution Architect can do almost anything technical and are valuable across a wide array of enterprise-wide projects and initiatives. The Solution Architect works on large and complex IT architecture projects with full competency. He/ she is responsible for major architecture deliverables for enterprise-wide programmes and LOB pro...

There are 3 main concerns that performance testing seeks to address. These concerns form the acceptance criteria or metrices for the tests. They are: User concern – Response Time Business concern - Throughput. E.g. requests/ sec; calls/ day System concern - Resource Utilisation (often overlooked). E.g. processor & memory utilisation; disk I/O; network I/O Purpose of Performance Testing Depending on what has been done, performance testing helps in assessing: release readiness infrastructure adequacy software performance performance tuning efficiency Some Definitions Performance Target = Performance Goals Performance Requirements = contractual obligation, SLAs that cannot be compromised. = Performance Thresholds Workload = Stimulus applied. E.g. number of users, concurrent users, data volume, transaction volume. Types of Performance Testing Load Testing - testing within anticipated production load/ volume Stress Testing - testing beyond expected production load/ v...

Some food for thought. Consider the following: Prefer BASE over ACID transactions Prefer asynchronous over synchronous transactions Keeping state is expensive Considering database sharding ( highscalability , codefutures , Pros and Cons ) by data, by transaction or by customer but avoid premature optimisation Design the system for automated rollback Create isolative structures; share nothing; such that nothing crosses the swimlanes Design systems for failure Create idempotent services where possible Database sharding requires changes in mindset: Tables may need to be denormalised to optimise sharding (as well as to workaround cross-shard joins/ queries) Scale-out instead of scale-up Do away with replication where possible Different sharding schemes are: Vertical partitioning – sometimes known as functional or feature partitioning where data relating to certain entities are grouped together. Different functions or features are put onto different shards. Rang...

Differences amongst IaaS, PaaS, SaaS   IaaS PaaS SaaS Application     provided Runtime   provided provided Database   provided provided Operating System   provided provided Virtualisation provided provided provided Server provided provided provided Storage provided provided provided Network provided provided provided

“Architecture expresses the intent, what we want to do; Implementation expresses the technologies used.” IT Architecture Body of Knowledge is summarised as follows: The Software Architecture determines flexibility The Infrastructure Architecture determines adaptability The Information Architecture forms awareness The Business Architecture helps productivity

The main concepts for designing CEP systems are captured in the following UML class diagram:

Meta-model for a Organisation Vision – a statement by company officials about the broad purpose of the company Strategy – grouping of plans that supports achieving company goals Goal – a specific objective that is measurable and achievable within the planning horizon Programme – provides the rules that govern initiatives and projects Project – a work effort that has defined start and end points. It consumes resources and generate values (e.g. cost reductions, revenue growth, benefits, etc.) Business Function – a set of procedures or activities that delivers products, services or support control of the company Capability – due to the business functions, the company is able to perform tasks, deliver services and develop products that sets itself apart from competitors Business Process – a procedure or activity that uses resources input to generate a measurable output. A group of these creates a function Application – a computerised system supporting automation for business proc...

Requirements, especially non-functional ones, need to be SMART: S pecific M easurable A ttainable (Achievable, Actionable, Appropriate) R ealisable (Realistic) T ime-bound (Timely, Traceable) For performance requirements (should these be more correctly called performance goals ?), the following aspects are most relevant: Response time – how fast the system responses to requests Throughput handling – how many requests the system can handle Concurrency – how many threads/ users can operate simultaneously To specify performance requirements, consider the following: Expected daily load: number of invocations/ day. E.g. System is expected to receive (on average) 500 requests everyday Expected peak load: max invocations / day. E.g. System is expected to hit a peak load of 1000 requests on certain days (Maximum) Response Time: invocations need to respond within certain timeframe. E.g. System needs to respond to requests within 5 seconds The last point (response time) deserves fu...

The quality attributes (*-ities) can be categoried according to the following (from SEI): Design-time Modifiability Maintainability Reusability Portability Testability Run-time Performance-predictability Security Reliability Availability Scalability Interoperability Through-put Capacity Operational Usability Supportability Configurability Sustainability Buildability

Broad Categorisation of required skills Technical acumen and technology application Software Application platform & Operating Systems Middleware Database Domain knowledge Vertical – industry-specific, e.g. banking, insurance, telecommunication, legal, government Horizontal – cross-industry, e.g. workflow, document management, search, SOA Conceptualisation skills – able to conceptualise, describe and communicate the solution Leverage patterns – don’t reinvent the wheel! Manage relationships – need to manage stakeholders, vendors, providers, partners Leadership skills – able to garner support/ buy-in Communication and listening skills Strategic skills – able to zoom-in and zoom-out Experience in Processes Solution architecture Systems engineering Software engineering Programme/ Project management Role is to: Develop the overall vision Execute the vision into the desired solution Organise the development effort Implement strategic IT programmes Toolbox Requir...

Emerging Technologies Cloud – for starters, for for non-business critical functions like CRM, email, collaboration Collaboration User experience Data deluge – what to do with so much data? Top Business Prioritises Business process improvement Reduce enterprise costs Increase use of information & analytics Improve workforce effectiveness Attract & retain new customers Business Intelligence State of Business Intelligence today – 20% of users in an organisation have BI in place; 80% of users do not. BI Solution Architecture Source system ETL processes Data Warehouse solution (ODS, Staging, Analysis cubes) Report presentation Microsoft Business Intelligence Strategy Familiarity – Office integration Collaborative – SharePoint integration Manageability – MS SQL Data Warehouse Trends in Data-Warehouse Increase in volume Reduce cost Adoption of appliances Move into MPP Desire for real-time analytics Realisation of the importance of data quality Microsoft...

Take the example of our customer-fronting website that requires integration to a 3rd party payment gateway. We have a situation where there are 2 distinct websites, one dependent on the other to conclude the business process/ flow as well as some form of SLA from the 3rd party. In essence, we do not trust the 3rd party payment gateway with our livelihood. How then do we integrate the websites together? Here are some of the items we desire: the 3rd party payment gateway appears integrated to our website security and regulatory requirements are met customers enjoy the entire website interaction integration failure are monitored or at a minimum, logged customers know what to do in the event of a failure (recovery or mitigation) The current situation is that: it is relatively easy to integrate with the 3rd party payment gateway, but the payment gateway further integrates with other banks’ websites for 3DS Assuming such a web flow: our customer-fronting website (order informatio...

What is an ODS? An environment: where data from different OLTP databases is integrated which provides a view of enterprise data that addresses operational challenges across more than one business function Characteristics of ODS: subject-oriented - catered to specific function or application (customer-centricity, risk management) integrated - from multiple legacy systems or new and legacy systems timely - data is continuously/ frequently being updated, typically more frequently than daily current - data is typically current with little history detailed - data is sufficiently detailed; not only at a summarized level central version of reference data ODS should be a separate data store from the data warehouse. Difference between ODS and Data Warehouse ODS DW Data Currency Current/ near-current Historical snapshot Data Loading Insert/ Update/ Deletion allowed Only loaded

It is a great idea to write a System Architecture & Design (SAD) Trade-off Document. The format is tabular and will look like the following: Module/ category Issue description Possible alternatives Decision Decision rationale Traded-off attributes Traded-in attributes Consequence/ constraints introduced Some examples of traded attributes are listed in the following table: Category Attribute System Performance Reliability - ability of the system to maintain operating over time (MTTF) Performance - responsiveness of the system to stimuli or events as well as throughput of the system System Control Maintainability - ease with which a system can be modified to correct faults, improve performance, or adapt to changing environment Data timeliness - data latency for information flowing into and out of the system Security - measure of system's ability to resist unauthorised access or DOS Supportability - ease with which a system can be maintained operationally Testa...

Types of integration Data-level Share data; not behaviour Minimal change (if any) to both source and target systems Can be database or file-based ETL File-data transfer Direct database access Bypass business logic (may need to duplicate) Overall data integrity may be compromised If writable, may lead to data corruption and referential integrity violations Capabilities include Data transformation Data validation Data access Schema definition Mapping Schema recognition Application-level Share functionality – business logic Based on API Composite applications Business Process level Share business processes Specified using BPMN Glued together using BPEL(4WS) and BPML Start by defining business processes; then specify logical integration within it Capabilities include: Rules processing Business transaction management Workflow Orchestration Event processing Schedule Presentation Share views Using a portal Non-invasive Application Integration methods Web services - Applicatio...