Choosing Randomness

An argument for selective randomness as the rational response to the structure of determined and undetermined experience

Ameet Kallarou

Abstract

Probabilistic determinism, as the term is used in philosophy of science, describes systems whose behavior is governed by deterministic laws but whose outcomes are irreducibly probabilistic. This paper argues that two forces, determinism and randomness, jointly account for the structure of human experience, and that their distinct roles provide the foundation for a theory of human development. The deterministic dimension tends to compound existing exposure and narrow the range of futures available to an agent. Randomness is the only force capable of widening that range, by introducing inputs the agent's own prior structure could not have generated. A rational agent who understands this structure should therefore orient deliberately toward increasing their exposure to randomness, not as a rejection of intention but as its necessary complement. This orientation, which the paper terms selective randomness, has consequences for how a life should be lived that the philosophical tradition has left largely unexplored.

I. Two Forces

Every system, biological, computational, or cosmic, stands in two relationships to its own causal history.

The first is determinism. An input or event is deterministic relative to a system if it follows from that system's own prior states, according to that system's own transition rules. A chess engine's next move is deterministic relative to the engine: it follows necessarily from the board state and the engine's evaluation function. A habit is deterministic relative to a person in the same sense: it follows from accumulated prior experience operating through whatever cognitive machinery produces behavior from precedent.

The second is randomness. An input or event is random relative to a system if it cannot be derived from that system's own prior states and rules, however precisely those states and rules are known. A pseudorandom number generated by a deterministic algorithm running on a computer is fully determined relative to the computer's transition rules. It is random relative to a neural network being trained using that number as a dropout mask, because the network has no access to the generator's internal state and no way to derive the number from anything in its own training history. The number is, from the network's perspective, exactly as unpredictable and exactly as decoupled from its own prior structure as a number drawn from any other source would be.

Whether the universe is ultimately, ontologically indeterministic at its most fundamental level is an open question in physics. If reality turns out to be ontologically indeterministic at bottom, that indeterminism constitutes one further source of randomness relative to any embedded system. If reality turns out to be fully deterministic at the most fundamental level, chaos, complexity, and the sheer scale of a system's external environment relative to a single embedded agent already guarantee that an enormous range of inputs will be random relative to that agent, in the full and relevant sense, regardless of what is happening at the deepest layer of physical law. For any sufficiently bounded system embedded in a sufficiently larger and more complex environment, the overwhelming majority of relevant inputs will be random relative to that system, whatever the deepest metaphysics of the universe turns out to be.

Humans possess awareness, the capacity to model their situation, anticipate consequences, and act on those models. That awareness is itself a product of the agent's causal history and operates within it. Consciousness does not introduce a third category standing outside the deterministic-relative-to-self and random-relative-to-self distinction. What it introduces, as this paper will argue, is the capacity to engage with the structure of that distinction in a substantial way.

II. The Narrowing Force

The standard philosophical response to determinism has been to ask whether free will survives it. Compatibilists, from Hume through Kant to Dennett, have argued that meaningful agency is possible even in a fully deterministic universe, provided freedom is defined as acting in accordance with one's own reasons and desires rather than external compulsion. Hard determinists deny this. Libertarians about free will have historically invoked quantum indeterminacy as a possible gap through which a different kind of agency might enter, an invocation that faces its own well-known difficulty, since an event uncaused by an agent's reasons is not obviously more attributable to that agent than an event fully caused by them.

This debate has been productive. It has also drawn attention away from a more tractable question: what does the structure of an agent's relationship to its own determining history actually do to that agent over time?

The deterministic dimension of that structure, operating without interruption by inputs random relative to the system, narrows.

Your experiences to date constitute your model of the world. That model determines what you notice in your environment. What you notice determines what you reach for. What you reach for becomes your next experience, which updates the model. The loop is self-referential and self-limiting. Each iteration tends to confirm and deepen what is already there. You become more capable, more efficient, more refined, within the domain your prior exposures have defined. Simultaneously, the territory you cannot yet imagine contracts.

Daniel Kahneman's two-system model of cognition maps onto this structure directly. System 1, fast, automatic, pattern-matching, operates by recognizing configurations encountered before. System 2, slow, deliberate, effortful, reasons forward from the model System 1 has built. Both are powerful. What they share, and what neither can escape, is that both operate entirely within the accumulated structure of prior experience, which is to say entirely within what is deterministic relative to the agent. The fastest intuition and the most careful reasoning begin and end inside the same identity. The frontier of what cannot yet be imagined stays fixed.

Socrates pressed on the logical structure of this problem in the Meno. How do you search for something you do not know? If you lack the concept, you cannot form the intention to seek it. If you cannot recognize the thing when encountered, you cannot confirm you have found it. The paradox of inquiry, that novelty is unreachable by directed search alone, is not merely a puzzle about epistemology. It is a structural feature of any system that operates purely by extending what is already deterministic relative to it.

Charles Sanders Peirce identified what he called abduction, inference to the best explanation, as a third form of reasoning alongside deduction and induction. Abduction generates new hypotheses rather than merely extending existing ones. But even abduction operates within the conceptual vocabulary available to the reasoner. The hypothesis space is bounded by prior exposure. An agent can recombine what is already deterministic relative to it. It cannot conjure what has never been random relative to it.

Left entirely to its own momentum, the deterministic dimension of experience compounds existing exposure without generating the conditions for its own transcendence.

III. The Opening Force

Randomness does something categorically different.

It introduces inputs that the agent's own causal history could not have generated internally. When an unexpected event enters a life, when a person collides with something outside their existing model, the agent's history receives material it could not have derived from itself. New material produces new effects. The space of futures available to the agent widens.

Evolution instantiates this principle as the primary engine of biological complexity. Random mutation, random relative to the organism because no feature of the organism's prior structure selects for or anticipates a particular mutation, produces variation. Natural selection then acts on that variation, preserving what functions and discarding what does not. The result, over geological time, is adaptive complexity that no process confined to extending the organism's existing structure could have approached. Remove the randomness and the variation disappears. Remove the variation and selection has nothing to act on. Variation first. Direction second.

William James, in the chapter on the Will in The Principles of Psychology, argued that the role of voluntary attention is not to generate new ideas but to sustain and prolong an idea's presence in consciousness once it has arisen, holding it in place against competing claims on attention long enough for it to develop effects. The origin of a new idea is essentially random relative to the thinker; what happens after its arrival, on James's account, is where the will's deliberate work occurs.

Nassim Taleb, in Antifragile, extends a structurally related intuition into a general principle about complex systems: some systems gain from disorder, volatility, and inputs random relative to themselves, rather than being merely harmed by them. Exposure to stressors and surprises produces not just resilience but growth beyond the original baseline.

Contemporary machine learning formalizes a precise version of the same structure. Neural networks trained by pure optimization, finding and reinforcing the pathways that minimize error on known data, reliably overfit. They become extraordinarily capable within their training distribution and brittle outside it. The standard remedy is dropout: the deactivation of neurons at positions selected by a pseudorandom number generator, fully deterministic at the level of the generating algorithm, but random relative to the network being trained, which has no access to the generator's internal state and no way to derive its outputs from anything in the network's own structure. Injecting inputs random relative to an optimizing system produces more robust and general capability than optimization alone. This is a mathematically formalized and empirically validated result, replicated across virtually every domain of modern machine learning.

Reinforcement learning makes the same point from a different angle. Any agent that exploits only what it already knows, always taking the action with the highest known expected value, gets permanently trapped in local optima. Every serious reinforcement learning system requires a deliberate exploration mechanism: a structured willingness to try actions whose outcomes are not derivable from the agent's current model, accepting short-term variance in exchange for the possibility of discovering better strategies. The formal study of the exploration-exploitation tradeoff is, at its core, the mathematical analysis of when and how much input random relative to the learning system to inject into that system. The field's conclusion, hard-won across decades of research, is that no fixed level of exploitation is optimal. Ongoing exploration is a structural requirement of any system that aims to keep developing.

When an input random relative to a person's existing model enters that person's causal history, it creates new exposure. Exposure is the raw material the deterministic loop runs on, the substance that feeds the model, shapes what is noticed, determines what becomes thinkable. More exposure means a wider model. A wider model means a broader space of available futures. The deterministic dimension of experience compresses the cone; randomness widens it from outside the system's own prior structure.

IV. Deliberate Action and the Spectrum of Variance

An objection arises naturally at this point. Does deliberate action not also widen the cone? When a person chooses to travel to an unfamiliar country where they know no one and don't speak the language, they are making a decision, and yet the experience that follows will be saturated with unpredictable events relative to their existing model. New people, unexpected situations, encounters with no analog in prior experience. The specific outcomes were not chosen. Only the context was.

Deliberate choices exist on a spectrum of variance. At one end are choices whose outcomes are highly derivable from the agent's existing model, more of what has already been done, in domains already well understood, with people already known. These choices deepen existing exposure without substantially widening what is random relative to the agent. At the other end are choices that deliberately court inputs the agent's model cannot derive in advance: entering environments where the rules are unfamiliar, where existing competencies don't transfer cleanly, where the agent's history encounters genuinely novel conditions. These choices are deliberate, but their mechanism of effect is essentially the same as randomness, they introduce inputs the existing model could not have specified or anticipated.

The distinction that matters is between variance-minimizing choices and variance-seeking ones. Deliberate action and randomness turn out to be the wrong axis entirely. Optimization, the deliberate refinement of existing capability in known domains, closes the loop. Variance-seeking, the deliberate choice of contexts where outcomes are not derivable from the existing model, opens it, even when the initial decision is fully conscious and intentional. In this sense, some deliberate choices are themselves a form of randomness-seeking behavior.

What a process random relative to the agent's selection mechanism adds, beyond what variance-seeking deliberate choice provides, is decoupling from that selection mechanism itself. Even the most adventurous deliberate choice is still a choice, made from within the existing model, toward options the model can already represent. A random process reaches outside that representational boundary entirely. The unexplored country chosen from a mental map is still chosen from a mental map. The outcome of a die roll is not.

Both have a role. Variance-seeking deliberate choices expand the cone within the territory the model can already gesture toward. Inputs random relative to the model expand it beyond what the model can gesture toward at all. Together they form the practical repertoire of anyone who takes the widening of the cone seriously.

V. The Problem of Undirected Randomness

If randomness is generatively necessary, the obvious question is why not simply maximize it. Why not abandon structure, routine, and intention entirely in favor of inputs random relative to oneself at every opportunity?

Undirected randomness produces variance without compounding. It exposes the system to an enormous range of inputs, but without the selective and integrative mechanism that converts exposure into growth, most of those inputs produce nothing durable. The evolutionary analogy holds here too: random mutation alone, without selection, produces drift, not complexity. It is the combination of randomness and selective pressure that generates adaptive structure. Chaos is not development.

The machine learning parallel is equally instructive. A neural network trained with too much dropout, too high a proportion of activations rendered random relative to the network at each step, fails to converge on anything. The injected unpredictability prevents the formation of stable representations. The optimal level of dropout sits well below maximum: enough randomness to prevent over-reliance on any particular pathway, not so much that stable learning becomes impossible.

The human equivalent of selection is integration: the deliberate cognitive and behavioral work of taking an input random relative to one's prior model seriously, following it, allowing it to reshape the model, and acting on what the reshaped model suggests. Without integration, such inputs remain anecdotes. With it, they become the material from which new capability, new understanding, and new ranges of possible action are constructed.

Selective randomness is therefore the core practical principle: the deliberate introduction of inputs random relative to the existing model, at a frequency and in domains chosen to maximize the probability that integration can occur. Enough randomness to widen the cone. Enough structure to process what the widening delivers.

The artist practices something like this instinctively. Brian Eno's Oblique Strategies, a deck of cards drawn at random to interrupt habitual creative decisions, does not dissolve the musical project into chaos. It introduces a constraint random relative to the musician's existing tendencies, which the musician must then integrate within the ongoing work. The result is neither the musician's uninterrupted intention nor pure accident, but a third thing that neither alone could produce. David Bowie's cut-up technique, John Cage's use of the I Ching as a compositional tool, are instances of the same structure: bounded randomness as a generative input into a directed process.

These practices were largely discovered through intuition and artistic necessity. They are not merely creative techniques but instances of a general principle: selective randomness is the rational response to the structure of an agent's relationship to its own determining history.

VI. Consciousness and Structural Awareness

A question lingers about what consciousness contributes if an agent's history is continuous and randomness operates as an input from outside that history. Is the agent doing anything at all when practicing selective randomness, or merely allowing itself to be acted upon?

What consciousness provides is structural awareness: the capacity to model the system one is embedded in, including the deterministic and random-relative-to-self dimensions of one's own experience, and to respond to that model rather than merely to immediate stimuli. This capacity is itself part of the agent's causal history, not a force that intervenes from outside it.

Most of the narrowing that happens in a life, most of the self-reinforcing loop of exposure and confirmation, happens without any awareness of the structure producing it. It simply unfolds. Structural awareness does not place the agent outside its own history. But it changes what that history is processing. An agent who understands the closed-loop tendency of deterministic experience, who recognizes the generative function of inputs random relative to their own model, carries a model of the world that now includes a representation of that structure. A model that includes that representation will, as a matter of the agent's own deterministic unfolding, produce different behavior than a model that does not.

Daniel Dennett, despite his thoroughgoing compatibilism, makes a related point about the evolutionary origins of agency: what distinguishes agents from mere mechanisms is the capacity to represent the future, model consequences, and act on those representations. The content of the representations matters. An agent who accurately models the structure of their own development will tend to develop differently than one who does not.

Structural awareness, then, operates as a cause within the agent's history, not outside it, one that, once introduced, propagates forward in ways that systematically expand rather than narrow what comes next. The awareness this paper is attempting to produce is itself, if the argument is correct, a widening of the cone.

VII. Toward a Practice

Philosophy at its best does not merely describe the world but changes what is possible within it. If the argument developed here is accepted, its practical implication follows directly.

The closed-loop tendency of the deterministic dimension of experience is not a fate to be lamented. It is a structural feature that, once understood, becomes navigable. It can be reliably interrupted by the deliberate introduction of inputs random relative to the agent's existing model, though it cannot be escaped or transcended. The mechanism is not complicated. The implications, compounded over time, are substantial.

What counts as a genuinely random input, for purposes of this practice, is defined by a single criterion: the outcome cannot be predicted or selected by the existing model. An input is random in the relevant sense when its introduction into the agent's causal history is decoupled from the preferences, habits, and patterns of thought that the existing model would otherwise generate. The practical elaboration of this principle, how to structure selective randomness, what constraints make it generative rather than disruptive, and how to integrate the inputs it produces, is the subject of other work.

VIII. Conclusion

Every system stands in two relationships to its own history: a deterministic relationship to whatever can be derived from its own prior states and rules, and a random relationship to whatever cannot. The deterministic dimension of human experience tends, without intervention, to compound what already exists, to narrow the cone of possible futures by looping through exposure already accumulated. Randomness is the force that widens the cone from outside an agent's own prior structure, feeding the deterministic loop with material it could not have produced on its own.

This is not a counsel of chaos. The generative function of randomness depends on integration, the deliberate engagement with what randomness delivers. Without integration, random exposure produces drift. With it, randomness becomes a primary engine of genuine development, not development as optimization of the existing self, but development as expansion of the range of selves that can subsequently be reached.

The philosophical tradition has spent enormous energy on the question of whether we are free. The more tractable and arguably more useful question is: given the structure of an agent's relationship to its own determining history, what should that agent do? The right response is not to seek freedom from causal history but to deliberately recruit the one category of input that can feed that history something new relative to itself.

Randomness is already everywhere in life. It always has been. Engaging with it deliberately, rather than passively absorbing whatever it delivers, represents one of the most consequential and least explored levers for expanding the range of what a human life can become.

This argument is developed practically in Dice Theory: A Guide to Selective Randomness.

References

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. Taleb, N. N. (2012). Antifragile: Things That Gain from Disorder. Random House. Dennett, D. C. (2003). Freedom Evolves. Viking. James, W. (1890). The Principles of Psychology, Vol. II, Ch. 26 ("Will"), section "Volitional effort is effort of attention." Henry Holt. Peirce, C. S. (1931–1958). Collected Papers. Harvard University Press. Plato. Meno. (Trans. G. M. A. Grube). Hackett Publishing. Eno, B. & Schmidt, P. (1975). Oblique Strategies. Self-published card deck. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). "Dropout: A Simple Way to Prevent Neural Networks from Overfitting." Journal of Machine Learning Research, 15, 1929–1958. Sutton, R. S. & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.