At the 4th Workshop for Heuristic and Exploratory Techniques held here at the lab on July 7 and 8, 15 colleagues gathered to discuss boundary testing.
Aside from presentations from Keith Stobie, Scott Barber, James Bach, and Robert Sabourin to name just a few, there was a breakout session that consisted of two teams. One team’s mission was to find ways to discover boundaries. Here is the (unscrubbed) brainstorm from a team consisting of Karen Johnson, James Bach, Scott Barber, Dawn Haynes, Henrik Andersson, David Gilbert, and Michael Bolton.
It’s heuristic and some will need explanation from the idea owner (some day), but I wanted you to see the raw list that was assembled in just 30 minutes of discussion. What ideas do they trigger for you?
- Probing
- Resource utilization
- Customs
- Removable drives
- Investigation of transitions
- Defocusing strategies
- Datatype converion
- Purposeful vs accidentally
- Trends and patterns
- Push every button, slide every lever
- Frenetic test execution
- Database model
- Confusable sets
- XML schema
- Dogpiling
- Grouping
- Intentional stress
- Tools (Nikto, Holodeck)
- Model validation
- Continuity and discontinuity
- Architecture
- Limits, extents, borders
- Communication paths
- Theories of error
- Licensing
- Reverse engineering tools
- Forensic tools
- Log files
- Ask developer
- Cookie expiration
- Test printer
- Magic numbers
- Taxonomy
- Linguistic analysis
- Misdeclared boundaries
- Space
- Terminator
- Mismatched intrfaces
- Finished
- Root and stem effects
- HTML schema
- Powers of 10 / powers of 2
- Opposite
- Embrace color
- Dissection
- Screen resolution
- Batch
- HW compatibility
- Transition to 3rd party
- Authorship
- DLLs missing
- Protocol
- Code analysis
- Unit tests
- API
- Code coverage methods
- Import old data
- States
- Complexity
- Events and triggers
- Exploratory search (for limits or transitions)
- Periodicity
- Scatter plots
- Concurrency
- Precision and fuzziness
- Traditional boundaries
- 1-3-7-11-more
- Experience
- Trend graphing
- Bisection search
- Round numbers
- Contracts
- Google it
- Reliability
- Segregation
- Sorting
- Visual discontinuity
- Failure analysis
- Blink testing
- OSI layers
- Comparison
- Brute cause analysis
- Exploit each bug
- Test something similar
- Trigger
- Array
- Alternate functional pathways
- “Monday morning” disease
- Incursion
- Intuition and instinct
- Reactivating dormant
- Malicious user
- Judging
- Anti-random
- Retrospective search
- Fuzz testing
- Marching and resonance
- Merging
- Decomp and recomp
- Composite data structures
- Watch the kids
- Max and min
- Switching
- Notice if you have a hard time finding a boundary
- Should and shouldn’t
- Built-in tests
- Internal vs. external
- Data replication
- Mirror
- Look in the spec
- Look in the secret spec
- Check the industry standard
- Controls (third-party)
- Leaks
- Author / authorship
- Configuration Management
- “Don’t do that”
- Encapsulation
- Sloppy
- Infinite
- Divide by 0
- Zero
- Java pitfalls
- “Impossible”
- “Not yet”
- Marketing claims
- Check Google groups
- System requirements
- Intended use
- Egg language
- Price
- Turn debug info on
- Monitoring
- Memory
- Multiple processors
- Transaction logging
- Selection
- Error codes
- Check bug taxonomy
- Roles and permission
- New build
