The Time of Brain Science and the Time of Physics

Buonomano and Rovelli (unpubl


The Time of Brain Science and the Time of Physics
In the laws of physics there is no present moment and no flow of time.According to general relativity there is no universal 'now' because time is subject to relativistic distortion.The universe is conceived as a four-dimensional spacetime entity, a block universe with no flow of time.By contrast, all of us experience a flow of time involving a present moment that moves smoothly on from one time coordinate to This is an open access article distributed under the terms of the CC BY 4.0 license.Perception (2023) DOI: 10.1163/22134468-bja10088 Downloaded from Brill.com 12/31/2023 10:14:18AM via Open Access.This is an open access article distributed under the terms of the CC BY 4.0 license.https://creativecommons.org/licenses/by/4.0/ the next.It is almost impossible to talk about anything without implicit assumptions about the flow and directionality of time.Any reference to action or causal processes implies a flow of those things through time.I cannot imagine any perception or experience that is not phenomenologically embodied in the present and does not involve flow through time.If the present moment and the experienced flow of time are illusions, they are illusions that cannot be denied.The contrast between the block universe hypothesis and the phenomenology of perception and action runs so deep as to be almost a contradiction, and it is not clear how it can be resolved.

Timing & Time
Two recent papers (Buonomano & Rovelli, 2021;Gruber et al., 2022) have addressed that problem, drawing on physics, brain science, and philosophy.(I use the term 'brain science' to encompass both neuroscience and psychology, and I write as someone who was a brain scientist before neuroscience existed as a discipline: I remember it emerging in the 1970s.)I do not have the expertise to comment on the physics or the philosophy in those papers.I think in terms of the brain that humans actually have, and how that brain could construct the experienced flow of time, and that is all.The arguments presented herein centre on two propositions about perceptual processing in the human brain.One is that the temporal resolution of perception is much coarser than the temporal resolution of reality, so the present moment as perceived is quite different from the hypothetical local present in physics.The other is that the present moment in perception is entirely divorced from the present in reality, insofar as it is a construct of information with time-marking information that labels the information in it as being in the present.Buonomano and Rovelli (2021) outlined discrepancies between the understanding of time in physics and in neuroscience.They agreed that the universe as understood under general relativity is compatible with the notion of a local present and local simultaneity, thereby preserving the possibility of a present moment in the universe, at least on a local scale.However, this still leaves issues that pertain to physics but not to neuroscience, and vice versa.For example, one question pertaining to neuroscience is 'What is the subjective "now", and what time window does it reflect -i.e., what is the "width" of the subjective now' (p. 7).Although they raised that issue, the authors did not discuss it, so I shall say a little more about it here.

The Present Moment in Physics and in Perception
The present in the brain is not the same as the present in the physical universe, if there is one.I am going to stick my neck out and say a little about what the present moment would be in physics if there was one.I hope a physicist will correct me if I have got this wrong.The fundamental grain of the universe is set by the Planck units of space and time.To say that one thing is at a different spatial location from another is, at bottom, to say that they have different Planck spatial coordinates.Similarly, to say that one thing is at a different temporal location from another is to say that they have different Planck time coordinates.Thus, if there was a 'present' in physics, it would have a time span of no more than one Planck time unit.When the present moment is at one Planck time coordinate, it cannot simultaneously be at any other: all of the others are either in the past or in the future.The Planck time is 10 -44 s. "It is generally assumed that this Planck time is the smallest possible interval of time" ('t Hooft & Vandoren, 2014).But it is an interval, the smallest resolution at which one thing can be said to be at a different time from another thing.
The present moment in the brain, in conscious experience or in information currently in perception, has a much coarser temporal resolution.Perhaps the best indicator of the temporal resolution of the present in the brain comes from research on nonsimultaneity judgement and temporal-order discrimination.Several studies have shown temporal-order discrimination thresholds of around 3-5 ms (Babkoff & Sutton, 1963;Fostick & Babkoff, 2013;Tadin et al., 2010;Wehrhahn & Rapf, 1992;Westheimer & McKee, 1977).There have been reports of nonsimultaneity or temporal-order discrimination thresholds of less than 1 ms (Henning & Gaskell, 1981;Zera & Green, 1993) but it is likely that, in those cases, discrimination was guided by nontemporal stimulus features (Henning & Gaskell, 1981).Some perceptual systems are sensitive to differences in the microsecond or even nanosecond range (Carr, 1993;Grothe, 2003), but the detected differences are used to construct a percept that does not represent temporal differentiation.To illustrate, the human auditory system can detect differences in arrival times of sounds to the ears on the microsecond scale (Grothe, 2003), but the end product is a temporally unitary percept of spatial localisation of the sound source, not of two sounds at different times.It does not resolve into a percept of two separate sounds until the time difference is ~5 ms (Wallach et al., 1949).
It is likely, therefore, that the minimum temporal resolution in human perception is about 5 ms.It can of course be much coarser than that: under some circumstances, temporal-order discrimination thresholds can be around 100 ms (Fink et al., 2006;Matthews & Welch, 2015;Nishikawa et al., 2015).These temporal-resolution thresholds define the present moment in perception.If two events are perceived as occurring at different times then only one of them can be in the perceived present at any given moment.Take two auditory tones each of 1 ms duration presented with a 5 ms gap between them.When the first tone is in the perceived present, the second one cannot be (even though information about it will be going through perceptual processing at that moment).When the second tone is in the perceived present, the first one cannot be (even though information about it may have been transferred to a memory store).Some researchers might argue that the perceived present has a coarser temporal resolution than that, but I have trouble understanding a perceived present where two tones can be perceived as both in the present together and yet also as being temporally ordered.
The millisecond scale is more than 40 orders of magnitude longer than the Planck time scale.The present moment in perception encompasses a vast multitude of Planck time units.According to physics, assuming there is a present moment, only one of those Planck time units can be the present at any given moment because they form a time-ordered series.There is therefore a profound incommensurability between the present moment of perception and the present moment of physics.That discrepancy can be understood as a problem of resolution.The unaided human eye cannot perceive things that are microscopic, such as bacteria, because the spatial dimensions of such things are below the limits on human visual acuity, even though they are many orders of magnitude above the Planck unit of space.Similarly, the human eye cannot resolve temporal differences between events on the microsecond scale even though those too are many orders of magnitude above the Planck unit of time.Temporal differentiation as perceived by us is not an objective feature of the world.It is a product of perceptual processing, a construct of information.
As such, it is a profoundly different kind of thing: while the world outside the brain (and of course the brain itself as well) consists of material particulars with spatial and temporal extension, the perceived world is information embodied in neural activity.If we perceive a stick that is 10 cm long, our percept of the stick is not itself 10 cm long.It is neural activity that encodes the length of the stick, along with all the other properties of the stick.In the same way, if an event is 500 ms long, our percept of the event is not itself 500 ms long.This is less obvious because, of course, light from the event enters the eye for its objective duration of 500 ms.After that, however, it is transformed into neural activity, and that activity encodes the duration of the event.As information, the encoding of the event's duration is a symbolic, semantic label, not a duration (Herzog et al., 2016;White, 2023).
The same applies to the present moment as perceived or experienced by us.Whether or not there is a present moment in the universe, and whether or not that present moment moves on through a series of Planck time units, the present moment that we perceive and experience is just a construct of information.It is, at any given moment (time coordinate), the set of current products of perceptual processing that are encoded as the present (White, 2020).Thus, we experience current perception as the present not because it is but because all of it is labelled as the present.The remainder of this paper develops that point, particularly in application to the information-gathering and utilisation system proposed by Hartle (2005) and further analysed by Gruber et al. (2022).Gruber et al. (2022) developed the notion of an information-gathering and utilization system (IGUS), which originated in work by Hartle (2005).In essence, an IGUS is an information-processing system in which, hypothetically, the present as experienced is a product of the functioning of the system.As defined by Hartle (2005) the term can encompass biological organisms such as human beings and constructed information-processing mechanisms such as (hypothetical) robots.Hartle interpreted past, present, and future as products of the ways in which IGUSes can process information.He proposed an IGUS, a robot, such that "there is a present at each instant along the robot's world line consisting of its most recently acquired data about its external environment" (p.101).This was developed in what amounts to an information-processing and storage model, illustrated in Fig. 1, which is based on Hartle (2005, fig.1).

The Perceived Present as a Collection of Time Markers
The IGUS is the system within the dotted rectangle.The box labelled P 0 holds a representation of the external world, defined as a contemporaneous card at the top of the stack.Consecutive temporal coordinates hold consecutive cards.For a given card N 1 held in P 0 , at the temporal coordinate where N 2 is at the top of the stack, N 2 takes the place of N 1 in P 0 and N 1 moves to P 1 .Cards at successive temporal coordinates are associated with information about cards being shifted along the chain to P 3 and thence to the bin, where they are lost to the system.The top part of the figure depicts a conscious (C) and nonconscious (U) processing system that generates predictions for future cards.Hartle argued that P 0 can be defined as the perceived present in that it holds information about the card at the top of the stack at that temporal coordinate, P 1 , P 2 , and P 3 can be defined as the experienced or remembered past, in that they hold information about cards at earlier temporal coordinates, and C has a notion of the future in terms of the predictions it generates.
The claim, therefore, is that the IGUS depicted in Fig. 1 generates perceived or experienced present, past, and future, all within the assumption of a block universe.But it can be argued that that claim is not correct.Since an IGUS is a simple hypothetical mechanism, there are few constraints on it so it is not useful for determining what is possible for actual information-processing systems such as the human brain.Even so, it cannot generate an experienced present moment.Let us accept the block universe assumption with its corollary that there is no present moment and no flow of time.Under that assumption, the IGUS depicted in Fig. 1 could generate conceptual knowledge about temporal coordinates, specifically that they form a linear ordered series.That would happen because information in the system is time-marked according to whether it is in P 0 , P 1 , P 2 , P 3 , or C.But it would only be conceptual knowledge.That is, it might be equivalent to the kind of conceptual knowledge a physicist has about time as the fourth dimension in a spacetime manifold, comprising Planck time units in an ordered series.It would not be like the subjective experience of the present and the flow of time that humans have.Hartle (2005) claimed that "[t]he 'flow of time' is the movement of information into the register of conscious focus and out again" (p.102).That cannot be the case.First, the word 'movement' presupposes the very flow of time that it is adduced to account for.In a block universe there is no movement.Second, if that movement could occur, it could not be apprehended by the IGUS because it concerns states of the IGUS at different moments of time, not different moments of time represented in the IGUS at the same time.All the information necessary for generating an idea of the flow of time must be there at one time, and indeed that is the point of the multiple memory locations in the IGUS.But the IGUS in Fig. 1 would not be capable of generating an idea of the flow of time.All it would have is the states of information and their respective time markers at any given temporal coordinate: there is nothing that ties them together or says that this moment in P 1 becomes that moment in P 0 .They are just isolated units of information.That helps us to see part of what is required for the human brain to have the experience of time that it does, and I shall return to that point when considering how the experience of time might be constructed in the human brain shortly.
In summary, under the block universe assumption there is no true present moment.In the IGUS there are contemporaneous sets of information about different moments, and that could support conceptual knowledge of temporal order.But it does not generate an experienced present or an experienced flow of time.It has information about time.That word 'about' is important, as will now be shown.Gruber et al. (2022) discussed possible variations on the IGUS idea that would generate different presents.For example, they designed a hypothetical IGUS (illustrated in their figure 3) that could hold information about the same moment twice.This would occur because the system assigns a single event to two locations in the internal time line of the IGUS.But it would still only be conceptual knowledge, and in fact it is not even certain that the IGUS would ever 'know' that information with the same time referent had occurred twice.More to the point, the IGUS would not be experiencing the same present twice.It would be experiencing the same information input twice, at two different times.One set of input might be (almost) in synchrony with events in the outside world and the other might be delayed, but that is all.The time that the information in conscious perception is about is different from the time at which conscious perception is.Manipulating delay in input to conscious perception does not manipulate the conscious present.Gruber et al. (2022) discussed some experiments in which attempts were made to manipulate the temporal experience of human participants.As an example, Gruber and Smith (2019) created a virtual-reality environment which could alternate between two different screens, one projecting information about the present and one projecting information about the recent past.Toggling between these screens disrupted the normal experienced flow of time to the point where participants experienced the past as being as real as the present and even becoming confused between the two.
But the experiment does not show what I think the authors claim that it shows (with apologies to the authors if I have misunderstood this).Here too, the key is the difference between the time at which the perceived present moment is, and the time which the information in the perceived present is about.The VR system in the study by Gruber et al. (2022) does not generate two different present moments.The dislocation is in, and only in, the time which the information in the perceived present is about.Sometimes it is about now, or as near to now as visual processing can get, other times it is about some time in the past.The participant continues to have one present moment that is experienced as proceeding through consecutive moments of time as usual.All that changes is that the input to the system is delayed by different amounts of time, much as would happen in a television broadcast that cut between live and pre-recorded segments.The viewer is not necessarily aware of what is live and what is pre-recorded, but that is irrelevant to the continuity of their experienced present.
A virtual-reality study run by Gruber et al. (2020) illustrates a different temporal dislocation.Observers wore virtual-reality apparatus on which the movements of a toy dog were screened.The feed of information was temporally dislocated, jumping back and forth in time.As a result, the observers became confused about whether they were seeing one dog or more than one: "They lost the experience of persistence" (Gruber et al., 2022, p. 5).This is not multiple present moments.The observer's present moment would be experienced as singular and continuous over time, as usual; in fact, the perceived temporal dislocation depends on that.The problem is with desynchronisation of information before it enters the system, not anything occurring within the system itself.The replay of a past event no more creates a different experienced present than watching a movie on a cinema screen does.It is just more confusing for the participant because of the immersive nature of VR.The time at which the participant's experienced present is is not the same as the time the information being presented to the participant is about.Manipulating the latter does not affect the former.
Incorrect synchronisation of perceptual information is not uncommon.The brain seems to assume that an outcome of an action, for example, happens with zero or minimal delay.Presenting an outcome of an action with a fixed delay results in adaptation that effectively resynchronises the events so that no delay is perceived.If the adapted system is then presented with an outcome that is not delayed, the resynchronisation persists and the perceived temporal order of those events is misperceived: the outcome of the action can be perceived as occurring before the action that caused it (Stetson et al., 2006).In this case it is not external input that is responsible for the temporal dislocation, it is an induced change to perceptual processing of that input.Nothing is going wrong with the experienced present.There is no duplication of the experienced present, no dislocation of the experienced present in time.There is just a loss of synchrony in the construction of perceptual information in perceptual processing, and that loss of synchrony gets fed into and registered in the one perceived present that we have.
Perceptual processing of input information takes time.The amount of time taken is not fixed but varies depending on the kind of information that is being processed as well as on resynchronisation processes as described above.The minimum time to emergence of a conscious percept is probably about 150 ms (Dembski et al., 2021;Förster et al., 2020;Koivisto & Revonsuo, 2010;Verleger, 2020).The perceived present, therefore, is running behind the objective present by at least 150 ms, and by an amount that can be variable beyond that.So the perceived present is a delayed and slightly scrambled representation of the objective present.But it does not feel like that: we experience it as the present, not as delayed in relation to the outside world.From the perceived present, perceptual information is transferred (through a bottleneck) to the first of a series of memory stores, a store called informational sensory memory (ISM) that retains information on the sub-second time scale (Coltheart, 1980;Jacob et al., 2013;Öğmen & Herzog, 2016;Sligte et al., 2010;Sperling, 1960).Perceptual information in ISM feels as though it is in the past, even though the fact of it being in ISM means that it is still there in the present.So why is it experienced as in the past, when it is just as much there now as the information in the perceived present is?
The answer is that, when perceptual processing is complete, all perceptual information is marked with time of occurrence.Time markers can be characterised as semantic labels that form part of the perceptual interpretation of the stimulus (White, 2023).Taking vision as an example, visual representations have spatial markers that locate them in perceived space.Such markers can persist while the object is occluded by another object and allow tracking of the object's motion while it is occluded (Lappe & Krekelberg, 1998).Time markers have an analogous function, locating perceptual objects in perceived time.One kind of time marker that has been proposed for information in perception and recent memory is a P.A. White / Timing & Time Perception (2023) Downloaded from Brill.com 12/31/2023 10:14:18AM via Open Access.This is an open access article distributed under the terms of the CC BY 4.0 license.https://creativecommons.org/licenses/by/4.0/time distance marker (White, 2021).This indicates how long ago, relative to the present, the object or features so marked occurred.All information in the perceived present is time-marked as being in the present and, as it is transferred to ISM, so the time distance marker on it changes in at least approximate correspondence to elapsed time, so that it is experienced as increasingly far back in the past.Different items of information that are in the system at the same time are differentiated by time markers for time of occurrence.That supports a representation of objects and events that includes temporal extension, on the subsecond scale.So, it seems to us that we perceive the present because all the information in the perceived present is time-marked as 'present' .The perceived present is the sum total of products of perceptual processing that are time-marked as 'present' at any given moment.It seems to us that there is a recent history on a very short time scale because information in ISM, which is just as much active in the system as information in the perceived present is, is time-marked as in the past by varying amounts.That is what gives us the experience of perceiving the present, and moreover the experience of perceiving the present in the context of recent history.

The Experienced Flow of Time as an Informational Structure
The time-marking idea can shed some light on the experienced flow of time.As we have seen, Hartle's contention that the appearance of a flow of time is actually movement of information into and out of the conscious register cannot be the case because only one moment can be experienced at a time and movement of information in and out of consciousness would necessarily occupy more time than that.Movement of information is not sufficient to generate an experienced flow of time.It is important to distinguish between actual time and represented time: the time distance markers discussed in the previous paragraph are features of represented time, but movement of information in and out of consciousness is a matter of real time.Only one moment exists at a time.To deal with this, the brain must hold a representation of a temporal series of events at a single moment, and must represent change and connection information also at that moment.So, it would be 'a became b became c became d' , where d is the present; or, perhaps, 'd was preceded by c was preceded by b was preceded by a' .It is not actual temporally extended change that is perceived as the flow of time.It is an information structure existing at a moment that represents the recent past as changing.
The kind of thing that is required is illustrated in Fig. 2 which is a very simplified and schematic depiction of an information structure that can support perceived flow of time.The figure shows a division of the information structure into the perceived present and ISM.Objective time coordinates are indicated at the bottom of the figure.Each column is an information structure existing at a single moment of time, where 'moment' can be defined in terms of temporal-order discrimination thresholds.So, the column above m 1 exists for a moment and is then replaced by the column above m 2 , and so on.Objectively, the information content of the representation is updated at every successive moment by new input into the perceived present and loss of old information that has reached the temporal boundary of the memory store or has decayed before getting there.The column above m 1 shows, in very simplified form, the state of information in the system at one moment.Two key kinds of information are represented there.One is time distance information (t 0 , t -1 ,…).This labels each informational representation in terms of how far it is from the perceived present.This would be supplemented by ordinal temporal information ('t 1 was before t 0 ' , and so on), which is not shown in the figure.The other key information is the word 'became' .This links each adjacent representation in the column and it says, in effect, 'A became B became C' , etc.).
It is not the change in the information structure from one objective moment to the next that accounts for the experienced flow of time.That is impossible because only one such state exists at any given moment.For such a thing to give rise to an experienced flow of time, one would have to experience something that is temporally extended, i.e., the change from one state to the next, and that is not possible.What generates the experienced flow of time is the information structure at a single moment.Thus, the information structure in the column above m 1 says that C became D (or that D was preceded by C), and so on.It should be noted that that connecting information is semantic, not verbal, just as spatial position markers are semantic, not verbal.Words have to be used to stand for it in the figure.The whole columnar structure is connected in that way.We experience the flow of time at a moment because it is represented in the information structure that exists at that moment.Whether persistence, change, and the flow of time are real or not is debatable, but there are perceptual constructions of persistence, change, and the flow of time, and those perceptual constructions may or may not correspond to the way the universe is.

Conclusion
The brain exists in spacetime just as everything else does.But the perceptual experience of time is a construct of information in the brain and its features cannot be assumed to tell us anything about time in the universe at large.In particular, all of what seems most fundamental to our experience of time is in fact time-marking information attached to information generated in perceptual processing: the perceived present, the recent perceived past, and the experienced passage of time, are all constructs of time markers and not informative about the physical nature of time.

Figure 1 .
Figure1.Information-gathering and utilization system (IGUS) proposed byHartle (2005).P 0 to P 3 are memory stores.The schema is a model of the external environment utilized for generation of behaviour.

Figure 2 .
Figure 2. Information structure for a perceived series of events, A to G. Four information structures are shown, one for each of four successive moments of objective time.