Computational Engineering / Optimisation algorithms
Deterministic and stochastic methods
Downhill methods, methods that sometimes go uphill, and why the second kind exists at all.
Lesson review: expert verification remains open
This lesson has received a full-context editorial review and a check of its technical explanations. Selected arithmetic has been checked separately. This is not independent engineering sign-off.
Checks needed for project use
- A computational engineering reviewer must check algorithms, convergence claims, statistical assumptions and numerical examples beyond the cases explicitly recorded.
- Reproduce quoted algorithm evaluation counts and reference minima from the actual seeded lab before treating them as verified benchmark results.
These are educational models and exercises. Check their assumptions before using the results. The full technical content and every diagram have not yet been independently verified.
Concrete and steel calculations retain their stated first-generation Eurocode teaching basis. Use the standards and amendments specified for your project; do not mix generations without an appropriate design basis.
Read the lesson register and outstanding review notesWhy this matters
Deterministic methods repeat the same procedure from the same inputs. Stochastic methods use random choices, usually with a recorded seed. Both can be local or global in intent; suitability depends on smoothness, constraints, discrete variables and the available evaluation budget.
By the end of this lesson you should be able to
- Describe gradient descent and say precisely what it cannot do
- Explain the acceptance test in simulated annealing
- Describe a genetic algorithm's population, selection and diversity
- Compare methods on an equal evaluation budget
Deterministic methods
Gradient methods use derivative information to improve a candidate. Their guarantees depend on the problem and algorithm; a local method on a nonconvex problem may converge to a suboptimal stationary point. Setting a derivative to zero identifies candidates, not automatically minima.
For a single catalogue choice, testing sections in objective order and taking the first feasible one can establish the best catalogue option if every earlier candidate has been checked against all requirements. This is ordered enumeration, not gradient descent. Coupled member choices need a broader search.
Stochastic methods
Simulated annealing can accept an uphill move with probability exp(−Δ/T), with suitable scaling and a cooling schedule. A genetic algorithm maintains and modifies a population; particle swarm uses candidate and group information to guide movement. Their parameters affect exploration differently and are not interchangeable. None guarantees a global answer within an arbitrary finite budget.
Comparing methods
Use the same problem, feasibility criteria and comparable budgets. Report objective evaluations, derivative costs and elapsed time where relevant. Repeat stochastic runs with recorded seeds and examine variation, rather than choosing a method from one favourable run.
Worked example
The same method on two landscapes
Given
- Two two-variable landscapes: one smooth with a single basin, one rugged with several
- Gradient descent started from the same point (1, 5) on both
- A dense reference sweep of the rugged landscape found a best sampled value of approximately 1.150
Find
What gradient descent finds on each, and what that establishes
Predict first
A gradient-based optimisation of a steel frame is run three times from three different starting designs and returns three different answers. What does that tell you?
Practice
A simulated annealing step proposes a design 0.5 units worse than the current one, at a temperature of 2. What is the probability it is accepted? Use exp(−Δ/T) and give the probability to three decimal places.
Practice
A genetic algorithm starts with 30 individuals and runs 40 further generations. Each generation retains 2 elites without re-evaluation and evaluates 28 new individuals. How many objective evaluations are used, including the initial population?
Check yourself
A gradient method finds the same answer every time it is run. Is that an advantage?
Summary
- A gradient method assumes the local slope is a guide to the answer; a stochastic one assumes it is not
- Annealing's acceptance test is one line and it is the whole method
- A genetic algorithm's diversity is what lets it hold two basins at once
- Exploration and exploitation compete, and every method's control knob is the same knob
- Compare on evaluations, because in structural optimisation one evaluation is one analysis