SOFTware is HARD

Life is full of constraints. It seems like you can never simply "have your cake and eat it". In Product Development , we have the following Project Triangle: Good - quality of the delivered product Fast – time taken to deliver the product Cheap – cost of designing/ developing/ delivering the final product We are told to pick any two! In System Development , the Project Management Triangle follows: Scope – the set of functionality for the system or the work to be performed Schedule – time taken to complete the project Cost – cost of implementing/ delivering the final system We are told to pick any two! A variant in System Development, a Project Management Triangle (more like a Square) follows: Scope – the set of functionality for the system Schedule – time taken to complete the project Cost – cost of implementing/ delivering the final system Performance – the ability of the system to take load and be responsive We are told to pick any three! For distributed systems u...

Machineries need oiling after some time; building structure deteriorates due to natural forces; facades need to be repainted; the Golden Gate bridge needs to be repainted; vehicles need to be maintained; what does that mean for systems? I’m inclined to believe that software systems, like all other things, need to be constantly maintained, oiled, cleaned, before they deteriorate. How does deterioration look like? Deterioration may appear in these forms: systems become slower progressively systems crash/ become unavailable more often systems become more bloated (larger codebase, more storage space required, etc.) The system was originally deployed and tested fine so what went wrong? How does deterioration happen? It typically happens due to the following forces: user-base increased post-deployment to a number that was not intended/ tested for smart users found ways to use the system that was not originally intended for operation/ support team did not make it a point to upkeep t...

This wonderful metaphor by Ward Cunningham reminds us that doing things in a “quick and dirty” manner or taking shortcuts sets us for a debt which, when not repaid promptly, incurs interest in the future. When developing systems, enhancing existing ones, or even fixing bugs, we will typically arrive at a crossroad: should we take the time to do it right; or should we take shortcuts and deliver quickly? It is generally in the interest of the business folks to roll out changes quickly; while the development folks would choose to properly design, implement and test changes before delivery. Fortunately, It doesn’t apply to everything (e.g. code smells or design flaw).  It requires a deliberated decision to do something that is not sustainable in the long-term, but yields a short-term benefit. The result of which is not in the interest of the system's ongoing maintainability.   Examples of technical debt Some common examples include: hardcoding values in source code postpo...

Most popular logging frameworks (e.g. Log4J, SLF4J, Log4Net, EntLib) allow for several logging levels to be instrumented. A common question is: What logging levels are appropriate under what scenarios? The following lists some common log levels are their usage: Fatal - Service/ application/ process about to terminate. Unable to proceed with normal operation. Force a shutdown to prevent (further) data loss. Examples: No more system resource (OOM) Error - Unhandled or unexpected system error has occurred. Current operation/ thread to be aborted. Issue needs to be fixed. Still able to continue with normal operation (at least for other users/ sessions) otherwise. Examples: database deadlock Unexpected/ unhandled exception Runtime errors Transaction aborted Can't create file Warn - Things are generally working fine. Recoverable conditions. Transient environmental conditions happened. Something happened (but was handled) that may potentially ...

Let’s admit it: any form of system documentation is not up-to-date . The moment we start producing it, it is out-of-date. If that is the case, we should do with minimum (perhaps, lean) system documentation that is kept current. This begs the next question: How little is enough? If I imagine myself taking over a system from someone else, I believe the minimum/ lean system documentation should contain the following: Solution Design description of the high-level process flow description of the main modules/ services in the system what are the architecturally significant use cases or main functions of the system? what processes/ components make up the system? E.g. Are there web-based applications? Are there batch processes? what are the databases in use? What are the primary (entity) tables in use? what output is generated by the system? E.g. Are there printed output? Are there output for system integration? Are there messages (emails/ SMS) sent? what systems are integra...

With respect to the Weblogic server, there are several runtime states that are interesting. In addition, certain events/ commands lead to the transitions to other known states. I attempted to capture the state transitions and events into a UML state diagram for easy reference. States in brown are end-states while those in yellow are transitional.

There are 3 main concerns that performance testing seeks to address. These concerns form the acceptance criteria or metrices for the tests. They are: User concern – Response Time Business concern - Throughput. E.g. requests/ sec; calls/ day System concern - Resource Utilisation (often overlooked). E.g. processor & memory utilisation; disk I/O; network I/O Purpose of Performance Testing Depending on what has been done, performance testing helps in assessing: release readiness infrastructure adequacy software performance performance tuning efficiency Some Definitions Performance Target = Performance Goals Performance Requirements = contractual obligation, SLAs that cannot be compromised. = Performance Thresholds Workload = Stimulus applied. E.g. number of users, concurrent users, data volume, transaction volume. Types of Performance Testing Load Testing - testing within anticipated production load/ volume Stress Testing - testing beyond expected production load/ v...

I just came across an interesting article regarding Dangers of Micromanagement. See here . I've extracted an excerpt which I'm in total agreement with. The dangers are: Less risk taking, less initiative, wasted resources . Employees will learn that a micromanager is going to direct in such detail that they will learn to wait for direction in what to do and how to do it .In a micromanagement environment, employees often end up waiting to execute ; or they move forward only to be redirected by their manager, wasting valuable time and resources in the process. Less innovation. When people are being told what to do, there is little to no room for creativity or new solutions . The value of diverse thinking is lost because there is a feeling that mistakes are not acceptable. Micromanagement does not facilitate a "continual improvement" mind-set. My experiences and observations thus far has not differed from the above. I’ve worked for micro-managers. I’ve worked wi...

Some food for thought. Consider the following: Prefer BASE over ACID transactions Prefer asynchronous over synchronous transactions Keeping state is expensive Considering database sharding ( highscalability , codefutures , Pros and Cons ) by data, by transaction or by customer but avoid premature optimisation Design the system for automated rollback Create isolative structures; share nothing; such that nothing crosses the swimlanes Design systems for failure Create idempotent services where possible Database sharding requires changes in mindset: Tables may need to be denormalised to optimise sharding (as well as to workaround cross-shard joins/ queries) Scale-out instead of scale-up Do away with replication where possible Different sharding schemes are: Vertical partitioning – sometimes known as functional or feature partitioning where data relating to certain entities are grouped together. Different functions or features are put onto different shards. Rang...

Scenario Planning Consider STEP factors: S ociological T echnological E conomical P olitical Perform Impact Analysis (Impact vs. Probability as the axes of a graph) Construct scenarios Porter’s 5 Forces rivalry barrier to entry/ exit substitutes bargaining power of suppliers bargaining power of buyers Balance Scorecard financial perspective – revenue, costs, profits, EPS customer perspective – responsiveness, base customers, CRM, complaints, branding internal business perspective innovation & learning perspective SWOT analysis S trengths W eaknesses O pportunities T hreats