Drawing from Debord and Heidegger, this film examines our subjective experience of time throughout history. It reveals how the relentless pursuit of productivity has shaped society and our perceptions of time, creating a perpetual present driven by an autonomous economy and accelerated by surveillance capitalism. The film explores the societal obsession with busyness and advocates for reclaiming time and leisure to foster true human flourishing.

We stand on the brink of a precipice. In front of us lies a society driven for the sake of productivity, the mechanics increasingly directed by artificial intelligence to serve corporate interests. Behind us lies centuries of experiments in social organization that provide the historical and material foundation of society. Some failed with communism’s collapse in 1991, while the capitalist experiment shifted to the authoritarian ‘capitalism with Chinese characteristics’ in the east, or the more subtly coercive neoliberal experiment in the west. This experiment has raised many out of poverty, and yet has not improved the quality of life. As “We stand on the brink… we peer into the abyss…” (Poe, 2022).we peer into the abyss,¹ many grow sick with emergent This is not to say that these diseases are naturally occurring facts, nor that these diseases do not exist. They are really experienced diseases that are contingent upon existing conditions of neoliberal capitalism.social diseases—anxiety, depression, exhaustion, and burnout. These conditions are a symptom of a way of life that lacks direction. With the gradual shift to secular life, we’ve turned away from communal practices that provide spaces and times that lead to human flourishing. These practices included holidays in which contemplation, prayer, and meditative practices were observed— e.g.like Shabbat and Sundays. These were times allocated to prayer, creative play, leisure, reflection, rest, and quiet solitude that are no longer protected or made available to greater society. Liberal democracies have abandoned the practice of community-rest, pushing it into the private domain of the individual, but what they have lost is leisure and the communities that exist there. The position presented here is not a call to return to some golden past, or a renewed spiritual order, but a historical account of the perspectives that led to the loss of leisure. It explores a revitalization of times and spaces for human flourishing that were formerly protected by a spiritual order, but can equally be rooted in secular thought. I propose points of resistance to a dominant social order that seeks to continually capture the vestiges of free time.

At this juncture, society can collectively decide what the future will hold for humanity—we can determine what that looks like, what we’re building, and to what ends. If we do not rise to this challenge, then the instrumental power that has emerged out of neoliberal capitalism will decide our future for its own banal ends. Together we can decide if we want increases in productivity to appear in the form of profits for the few, or as surplus time allocated to the many—the latter of which can be used for leisure and the concrete activity of creating a brighter future for all.

1. Introduction
2. The Historicity of Time
3. Temporalities of Time
4. Instrumentarianism and the Perpetual Present
5. The Rise of the Zeitzombie
6. Leisure as Refusal: The Power of ‘No’
7. Bibliography

The Historicity of Time

’Time’ is natural, physical, and spatial; it exists independently of humanity. ‘History’, however, is humanities Bunyard 2018, p. 216.awareness of its existence within a temporal reality—only arises with the emergence of human beings,² and is shaped by humanity. The great arcs of history are determined by the rulers of their period, and individuals are capable (to a certain extent) of generating their own personal narratives within that period. Bunyard 2018, p. 4.History, therefore, is a process directed by human agents to shape their world, and create their own identities through that activity.²

In order to establish an understanding of the perception of time across history, I will begin with a periodization that draws from the philosophical anthropology of Guy Debord in See chapters V. and VI.The Society of the Spectacle.³ Debord presents a series of historical periods characterized by social structures that engender different perceptions of time called ‘temporalities’. The following table lists these distinct periods. Each age is mediated by a form of social organization, and that organization generates a corresponding temporality. I will describe each in turn.

Each age describes the dominant forces that structure daily life—the A zeitgeber is an external cue that entrains or synchronizes an organism’s biological rhythms. These are natural factors like light, temperature, seasons, but can include social cues like work schedules or gig apps.zeitgeber, which shape perceptions of time. There is overlap between ages, both temporally and spatially. An age is more keenly experienced in the city than in the countryside. The procession of one age to the next doesn’t entail that the previous age has ended, but that its’ forces have receded to the background—they become infrastructural rather than hegemonic. To illustrate, the mythic narratives that arose from the Age of Gods & Men remain with us in the form of stories and metaphors that provide us with rich cultural allegory. In another case, the Age of Kings had no definitive end, in fact, relics like the Queen of England still exist, but monarchy is less important these days, and thus has become merely cultural. Institutions like religion, which previously dominated social organization have similarly receded. In some sectors, secular life has completely subsumed spiritual life, and spiritual life is relegated to the private domain of the individual. The ages of the past are still with us, but they’ve become a given, received culture, one that is taken for granted—its issues and contentions are largely settled debate, but remain relevant. Past ages become social and cultural foundations—but just like a buildings’ foundation, they are more ossified and unchanging. The hegemonic age, however, dominates in its effects on daily life, it organizes social institutions, it’s open to continual evolution and debate, and is always changing and dynamic.

The Age of Gods & Men is the period before prehistory. It is accessible only through the archeological record and surviving oral traditions. This is a period in which nature recurs with time, the coming and going of the seasons and celestial bodies. This is a Mythic society prior to the mastery of language. Mythic society was a directly lived experience, created by their own direct actions, one in which all individuals directly participated—by necessity. The hegemonic temporality associated with this age was Cyclical time. Cyclical time shaped peoples understanding of time as they encountered naturally cyclical conditions, repeated along every moment of their nomadic journey with the seasons and according to the migratory patterns of wild game and local flora. The spatial modality of this period is characterized by territories that were undifferentiated and therefore uniform—that is to say, hunter-gatherers didn’t stay in a single territory long enough to shape their environment. This age applied to human societies until the invention of agriculture, animal husbandry, and the emergence of subsistence farming.

The Age of Cyclical Nature is characterized as a time that is proto-historical. When agriculture began, communities invested labor into the land, The Clearing (die Lichtung) is Heidegger’s concept of the open space where Dasein creates themselves.clearing⁴ it for planting and imbuing it with content. This attached the community to space, and therefore enclosed the community within cultivated property—one surrounded by a differentiated region of uncultivated lands. This was a shift from the nomadic seasonal cycle, returning people to undifferentiated but similar places and a familiar set of gestures in time (planting, harvesting, etc) that were attached to a single place. Agrarian society generated the spatial modality of “differentiated property” and “undifferentiated lands”. The transition from the pastoral nomadism of the Age of Gods & Men to settled agriculture marked the end of an idle and unattached freedom, and the beginning of labor. The Agrarian society and agricultural mode of production is governed by the rhythm of the seasons, and as such is the implementation of Cyclical time in its fullest development.

The Age of Kings emerged from the practice of permanent agricultural settlements, and the growth of cities.⁵ The Age of Kings is characterized by the division of social classes, each specializing in a different aspect of the total production of society: farmers, warriors, priests, and the nobility. The nobility organized social labor on the basis of the accumulation of wealth in the form of Surplus value during the Age of Kings was the total amount of crop remaining after farmers fed themselves and paid taxes.surplus value. This class appropriated the temporal surplus value that resulted from the exploitation of the farming class. Surplus time was invested into the development of a new form of power: the control of spatial territory and the development of history that was at the sole discretion of the nobility. The development of history is a new form of power controlled by the nobility over the direction, management, and recording of historical time. This is a power inaccessible to the laboring and agrarian classes. This power is most visible in the historical chronicle—a story narrating the events deemed most significant to the ruling class, and often the only recorded history until the invention of the printing press. The nobility used this instrument to establish the forward progression of their historical time by recording its past with writing. Mythical time is the realization of the historical chronicles of the nobility that has developed autonomously and as a separate sphere of abstracted reality from natural reality. This was the case with China and Egypt, who held a monopoly on the immortality of the soul, and the earliest of their famous dynasties are built upon Debord 2021.imaginary reconstructions of the past. The rulers of these empires, as the owners of the private property of history—protected by a mythical past, make use of myth to prove the legitimacy of their claim to rule. The intention is to be understood as the earthly execution of Mythic commandments would later be repurposed by religion to continue underwriting the control of power in the spiritual domain, best exemplified by the Holy Roman Empire.mythic commandments. The more they claimed historical ownership over time and tied it to their own mythical chronicles, the more they legitimated their control over power.

The development of mythical history began with the administration of kingdoms that went hand-in-hand with the invention of writing. Mythical history gives time an orientation, a direction, and imbues it with meaning and significance. The emergence of history necessitated a development beyond Mythical time to the temporality of Irreversible time. The linear trajectory of irreversible time presupposes endless economic development linked to the production of surplus value, the Lowercase c-capital in this instance can take the form of money, labor, and time.control of this value in the form of capital, and the political management of the resulting power. Irreversible time is oriented around the succession of the nobility, and its measure of progress is determined by the number of successful successions. The Age of Kings was a profoundly Historical society in that social organization was underwritten by history. It was during this age that history had increasing proximity to everyday existence but the possibility of individual participation grew more elusive. Individuals living in this historical society experienced lived moments as mediated through the nobility or God as facilitator, since the nobility had sole possession of The conception of duty has been a means used by the holders of power to induce others to live for the interest of their masters, rather than for their ownhow individuals’ time was used and allocated—so called ‘duty’.⁶ The nobility were the only ones to be in a position to have knowledge of, and experience the enjoyment of directly lived events, while those in the city spectate upon what is essentially a conflict over power—witnesses to a story that is not their own; which is to say, they were mere spectators, alienated from directing the most important events that define their lives, and destined to be forgotten. In order for the irreversible time of history to cohere in the collective memory of individuals, there needed to be participation in these officially recorded events (as it was their duty). The wealth accumulated by the nobility was expended on lavish feasts and festivals—best exemplified by the ‘bread & circuses’ comes from Juvenal, a Roman poet active in the Second Century AD. The phrase is a critique of the political class who deliver not public services or policy, but instead public diversions, distractions, or other means of appeasing the populace with trivial or base appeals to amusement or other simple pleasures that make no lasting change or improvements to society.bread & circuses of the Roman Empire⁷ that created fleeting moments of participatory lived experience mediated by the nobility. The major events of this period culminated in the Crusades.The resulting participation in these events is a recognition amongst individuals as the possessors of a unique present, a period defined by the richness of their own actions, and a home built by their own (mediated) experience. For members of the historical society, irreversible time truly exists, and within it they create memories of their own history as well as the emergence of a newfound fear of being forgotten into the oblivion of cyclical time. The spatial modality of this age was the control of territory, and the weapon of choice was, for the first time, the written word. Ironically, the power of the nobility was legitimated by claims (familial or fictional) to a mythical past. It was during this age that E.g. Judaism, Christianity, Islam, Buddhism.monotheistic religions developed and made a compromise between myth and history—one that bridged the gap between cyclical time (as it dominated the sphere of agricultural production), and the irreversible time that was the theater of politics. The religions that developed out of monotheism were the fundamental building blocks universally acknowledged as useful for building new historical societies based on irreversible time. It was during this age where one’s life was measured according to irreversible time, in the form of successive stages of life (childhood, adulthood, old-age), with the consideration of life as a voyage, as passage without return, in a world whose meaning culminates in the spatial elsewhere of According to the Augustinian Proclamation, the Catholic Church in Vatican City was the unitary physical space shared with Heaven (Augustine 1993).Heaven.⁸ Individuals found the fulfillment of their personal histories within the sphere of the ruling nobility. So, just as the nobility defined themselves by waging war over disputed power, individuals attached their personal historical chronicles to their rulers—thus, the knight and his personal story is entwined with the story of duty to his king. The Age of Kings applies to feudal civilizations in China, Egypt, Greece, Iran, Rome, Europe, Japan, and Latin America.

As long as agriculture remained the primary form of labor during the Age of Kings, cyclical time continued to constrain social life with tradition, which inhibited the development of irreversible time. It wasn’t until the rise of mercantilism that the bourgeoisie emerged, challenging the monopoly on irreversible time.³⁹ It was only then that the irreversible time of the bourgeois economy was brought to bear upon the remaining vestiges of cyclical time, eradicating it at every encounter across the globe. As the bourgeoisie gained control over irreversible time, their labor gradually became a project to transform historical conditions.³ The bourgeoisie was the first ruling class for which labor became a valuable commodity. With the gradual abolition of all nobility, social privileges, and titles, they recognize value only from the exploitation of labor, and have identified the control of the commoditized body of labor as their primary form of capital. These developments culminated in the emergence of the Age of Economic Production.

The Age of Economic Production is characterized by working conditions that were the dominant zeitgeber of daily life. This age emerged during the enlightenment and prefigured the industrial revolution with the inventions of the printing press and the clock tower. Irreversible time limited participation to the nobility up until the period of the The bourgeois revolutions include the English Civil War (1642-1651), the American War of Independence (1775-1799), the French Revolution (1789-1799), the European revolutions of 1830 and 1848, and the unification movements in Germany and Italy.bourgeois revolutions. During this age, irreversible time developed into its generalized form—the inevitable unfolding of events for themselves, crushing any individual in its path. This was the emergence of the temporality of Chronological time: a progression of time marked by the unfolding events of the commodity, and controlled its market imperatives. This is a time of clocks that is measurable, ordered, countable, and structured. The bourgeoisie, as rulers of the economy and specialists in the ownership of commodities, untethered history from the administration of the state and privileged the management of the economy. Increasing literacy and the historical power unleashed by the proliferation of the printing press democratized and diminished the hegemony of irreversible time. Individuals participated in the production and consumption of commodities, which was formerly limited to the nobility.³ It was during the later part of this age that capitalism developed across the globe, universalizing chronological time as a singular history that progresses the same everywhere at once. This is a time belonging to the globalized marketplace. The spatial modality of this age was migration, where migrant labor was the major factor in human movement. People lived where they worked, migrations were predicated on moving to locations with improved working conditions. Proximity to work, as well as citizenship and residential status determined access to employment. In the United States, young people moved from familial support networks to cities with better jobs. Space was a constraining factor in this age, where proximity to arts & culture limited what individuals could watch and listen to. The city was the locality of the arts. The conception of utopia during this age was increasingly secular and socially constructed, resulting in the development of liberal democracy and socialism. It was during this age that the concept of leisure emerged as the temporality of Consumable time. Consumable time is the colonizer of non-working time. It is relegated to a subservient role as a gift that presupposes labor rather than as equal to chronological time—it is free time that is legitimated by work, but not free in the sense of freedom. To maintain these complementary roles, consumable time finds itself laden with false attributions of value, and these moments are segmented into a sequence of artificially distinct events, which are merely undifferentiated moments of time perceived to be In consumable time, the weekend is perceived as more valuable than the weekdays, and the general sentiment that Mondays are worse than Fridays. These moments of consumable time are perceived to be more valuable since the worker is able to consume this time freely.more highly valuable.³

The Age of the Autonomous Spectacle is the ultimate realization of the The ‘End of History’ is a political and philosophical concept that supposes that a particular political, economic, or social system may develop that would constitute the end-point of humanity’s sociocultural evolution. A number of writers have argued that a particular system is the end of history including Thomas More in Utopia, G. W. F. Hegel, Karl Marx, Vladimir Solovyov, Alexandre Kojève, and Francis Fukuyama in his 1992 book, The End of History and the Last Man. Papaïoannou is more optimistic, saying “a time of [historical] arrest is not the end of time”, argues that it can contain the potential for new futures (Papaïoannou 2012, p. 97).End of History¹⁰¹¹—a refusal of the progression of any major historical divergence from our extant social organization and temporalities. It is an indefinite postponement of the future. This is an age characterized by commodities that exist for their own sake, external to any human desires, or even society itself. This is an age where the “Because value shapes the actions and interactions of individuals within capitalist society, and because such social activity takes place according to its needs… human agents become confronted and dominated by an antagonistic counter [the economy] that governs society in their stead.” (Bunyard 2018, p. 316, altered).economy becomes autonomous—organizing social activity² in order to reproduce itself.¹² This age was inaugurated with the publishing of Propaganda, was written by Edward Bernays in 1928, incorporated the literature from social science and psychology for the techniques of public communication for the manipulation of the masses. (Bernays 2005).Propaganda¹³ in 1928¹⁴, it developed with the ubiquity of television and radio. It was formally declared when Margret Thatcher announced “There is no alternative” (TINA) is a political slogan arguing that capitalism is the only viable system of social organization. It was used by Margaret Thatcher in a speech. Together with Ronald Reagan, they began the era of neoliberal austerity and laissez-faire globalization. “There is no alternative to capitalism”.¹⁵ , and echoed by David Cameron: The age finally materialized with the fall of the Berlin Wall, when all alternatives to capitalism ceased to exist.

The temporal modality of this age is Spectacular time, with the perception of time itself as a consumable commodity. Spectacular time is generalized across the globe as a repetition of the same day, a Groundhog Day¹⁶, spectacular time is a uniform and equal amount of time fully allocated to the production and consumption of commodities, and the reproduction of itself. This is a time that can only be organized by the specialized interest groups who own the production of commodities. Just as irreversible time was democratized during the Age of Economic Production, the Age of the Autonomous Spectacle democratizes chronological time by granting individuals the ability to create and consume commodities in the general economy (whereas it was once limited to the social elite). This is participation with the historical record, albeit limited to the production and consumption of commodities that increasingly becomes the totality of lived experience.

It is during this age that social media allowed individuals to participate in history The celebrity is the contemporary class that was formerly the role of the bourgeoisie, the ruling class, or the nobility—the appearance of the control of power and influence: The Influencer.as celebrity by ‘going viral’ either among their peers, or more broadly. This is an ephemeral participation in history however, as the attention—or gaze of the public—is distracted by the continual onslaught of spectacular memes. Individual subjects of the viral event are quickly subsumed by the global flows of the attention economy that expands autonomously according to market imperatives. This commoditized form of time segments moments into spectacular time and Pseudo-cyclical time. Spectacular time is the equivalent of work, and pseudo-cyclical time is its’ corresponding free-time. Pseudo-cyclical time is perceived to have more intrinsic value than working time—work is the so-called ‘real life’ the worker must endure. The consumption of these images (media broadly: internet, tv, news, magazines, apps, even academic publications) serve as further advertisements for all other commodities.Any surplus pseudo-cyclical time is increasingly allocated to the consumption of images, deepening the immersion in Spectacle is a pervasive system of mediation that alienates individuals from authentic social interactions and experiences. It is not merely about passive consumption of media but encompasses the entire fabric of society, including politics, culture, and everyday life where reality becomes fragmented and commodified, with images and representations serving as a substitute for genuine human relationships and directly lived experiences (Debord 2021).Spectacle. This time is the realm of the commodity, and the images are the medium with which these commodities act upon our consciousness most efficiently. This is the realm of the attention economy, a battlefield from which the Spectacle mounts its strongest attack. The ultimate goal of this attack is to maintain our attention upon the Spectacle itself, and to increase the time allocated to the consumption of spectacular events. Whereas power in the Age of Economic Production was determined by nation-states, who made their power legible by drawing territorial boundaries, and defining residents and aliens—the Age of the Autonomous Spectacle is one where power is determined by attention, and space is no longer relevant.

The invention of the smartphone is the key development (after the emergence of the internet) that established the ubiquity of spectacular time and cyberspace. The smartphone locates us on the internet all the time. We previously went to the place of the internet and took things away with us, but smartphones push us over a threshold where we need to opt-out of Named ‘cyberspace-time’, cf. Fisher 2020, T00:08:30.spectacular time. With the internet in our hands, in the ubiquitous non-place of cyberspace—we’ve lost the ability to opt-out.¹⁷ Airspace is a term coined by Kyle Chayka (Chayka 2016).Airspace¹⁸ is the spatial modality of this age: the universal realm of the global market, where local goods, cultures and the places themselves are made accessible to the market, each comparable and homogeneous to better conform to the ease of exchange—and therefore lacking any semblance of its once unique character.³ Airspace is a banalizing and homogenizing force across the globe—where the same tertiary services appear everywhere as workers travel the world demanding the same familiar experiences, creating a banal nowhere.¹⁸ This can be seen happening simultaneously, with the reproduction of the same coffee shops, stores, languages, restaurants, styles, and customs across the globe. The emergence of Airspace corresponds with the success of non-location companies like McDonalds, Starbucks, Airbnb, Uber & Lyft, Zoom and VPNs are technologies that negate space itself, as well as the need to travel.Zoom, and VPNs. Workers are no longer tied to their localities, cities, or even countries. Workers can move to their hometowns, move in with families, or move abroad in order to change their working conditions. In this age, the nation-state becomes a barrier to free trade, a barrier to the worker trying to optimize their working conditions. With the obsolescence of the nation-state, corporate power comes to the fore. Corporations are the dominant zeitgeber in the daily lives of individuals. They provide the infrastructure for wages, health care, perks, and even define when, where, and who people socialize with. The corporation is the predominant institution that organizes society. The rise of corporate sociality is built upon the graveyard of community. The ideal worker in the Age of the Autonomous Spectacle is the knowledge worker, the self-starter, who has paid for their own training and whose primary task is to produce their own psyche: an The achievement-subject refers to individuals who have transitioned from being obedience-subjects in a disciplinary society to entrepreneurs of themselves in an achievement society. These individuals are driven by a task-oriented compulsion towards constant productivity, self-improvement, and overachievement. They are guided by a positive social milieu that emphasizes “Yes, we can”. This condition ultimately leads to anxiety, exhaustion, burnout, and depression (Han 2015).achievement-subject¹⁹ who is expected to have little inputs and produce maximum outputs, with little management, and produce surplus value that consistently goes beyond ‘meets expectations’.

Temporalities of Time

The perception of time is a fluid modality of human thought. A temporality is a collective psychology that dominates the whole mass of society.²⁰ These psychologies shift across periods of time and space, much like an age, a temporality is more keenly felt in the city than in the country. One temporality comes to the fore as another recedes, but one is always more dominant. The dominant temporality of any society is determined by its stage of human social development. During the Age of Cyclical Nature, time was humanized and therefore any individual was able to participate in the historical events of their own time. After the emergence of irreversible time, the individual could no longer meaningfully participate in the lived historical events of their own period, as those events are no longer controlled by those at the base of society.

Cyclical time shaped peoples understanding of time as they encountered naturally cyclical seasons, and according to the patterns of local flora and fauna. It is a perception of time as the passage of the labors of a people in sync with naturally occurring transitions.

Mythical time is perceptually the same as Cyclical time, but underwritten by a claim to ownership of that time by a nobility that has taken ownership of the historical record.

Irreversible time is the perception of time as a linear series of events and developments occurring in the realm of war, economics, technology, knowledge, and yet limited to the personal chronicles of the nobility. It is qualitative. Irreversible time is a temporality whereby the ruling class records their personal histories with narratives—those personal events, conquests, wars, battles, and the administration of their kingdom. Irreversible time is the official time that is contested in clashes between foreign lands. Irreversible time is thus alien to ordinary individuals, something they don’t seek out and something from which they had thought they were protected from. These events mark changes in the historical record that provide a framework for individuals to understand their place within that history.

The early ages we’ve described each correspond to a single temporality. In the later two ages (the Age of Economic Production and the Age of the Autonomous Spectacle, respectively) two temporalities emerge that exist simultaneously. One of these temporalities always corresponds to the In Debordian terms, ‘time of production’ (Debord 2021).time spent working and one corresponds to valorized time (free time that is highly valued by individuals). Individuals experience these temporalities distinctly, and oscillate between them on a day-to-day basis.

Chronological time is linear and quantitative. It is segmented, measurable, ordered, and counted. It is the time of clocks, of hours, minutes, seconds, milliseconds.²¹ Chronological time is the Chronos, it is scheduled and planned in advance, knowable and quantitative.²² It is the time of work.³ Chronological time atomizes the moment, Equivalent time where every interval is equivalent in value and therefore indistinguishable as abstracted units of time, which are, in reality, merely their use value for exchangeability (Debord 2021).where every interval is equivalent in value and therefore indistinguishable.³ Chronological time has no duration, and produces no memory, it is unmemorable time but not forgotten—it is always inscribed into the historical record. Chronological time is understood as a sequence of the same uneventful ‘nows’ that repeat over and over again to the point where each working day is interchangeable with the next. Chronological time is the complement to consumable time, and the opposite of Lived time.

Debord uses the term ‘Pseudo-cyclical time’ when discussing the freely consumable time of the worker that is the compliment of time spent working.Consumable time is the time returned to social life as a by-product of the work accomplished during chronological time³. Consumable time is chronological time disguised so it may be consumed freely by workers. It exhibits the same characteristics, namely equally segmented, fungible units of time—the qualitative dimensions of which are suppressed in favor of quantitative equality. Since this time is the complementary by-product of work, it therefore maintains the hegemony and legitimacy of work. It is a commoditized version of Lived time.

Spectacular time is the ultimate realization of the End of History—a refusal of the progression of any major historical divergence or change to the dominant temporality. It changes our perception of time, Twitter is the perfect example of a ‘screaming low-level panic’ (Fisher, 2020).creating a panic-temporality.¹⁷ It enforces a state of constant anxiety akin to entrepreneurialism, a form of precarity where we are compelled to continually hustle and market ourselves even when we have stable jobs. This temporality appears across the globe as a repetition of the same day, a uniform and equal amount of time fully allocated to the production and consumption of commodities, and the reproduction of Spectacle. In spectacular time, we never have the time to sit back and think, to plan for the future—all time has been accounted for, and the future has been foreclosed. This is a time that can only be organized by the specialized interest groups who own the production of those commodities. Spectacular time is an objective religion with dogma, ritual, and the imposition of scripture as fact—these take the form of best practices, productivity optimizations, and quantified success, respectively. This isn’t a subjective religion that is lived and felt, but one that colonizes life. It is a religion that is submitted to from the external doctrines of the economy.

Pseudo-cyclical time is the complement to spectacular time, a time valorized for the ‘free consumption’ by the workers in the Age of the Autonomous Spectacle. Pseudo-cyclical time is experienced as cyclical time, returning to the The natural zeitgebers are the cyclical rhythms of cyclical time: day and night, the procession of the seasons, the intervals of farm work, the cycle of harvests, festivals, and holidays.zeitgebers that regulated the intervals of pre-industrial societies. It builds upon the vestiges of cyclical time as a foundation and generates new variations: day and night, weekly work and weekend rest, and the cycle of fashions, television shows, holidays, and periodic vacations. It is defined by the continuation of the rationality of the enlightenment combined with the demand for data that arose out of the attention economy. Pseudo-cyclical time is itself a consumable commodity, one which has combined all aspects of the social life which were formerly separate: private life, economic life, and political life. This combined time of contemporary society is the raw material to be consumed by the worker however they see fit, but only as inputs to a set of ever-expanding products on the market of socially controlled schedules of the attention economy.

There are two more temporalities, these are the unmemorable, the unaccounted for, and therefore the untheorized. These two temporal domains are the wilderness of time, at once wild sites of potential and yet vulnerable to commodification.

Reproductive time is the time used to reproduce the worker every day. This is the time required to acquire daily food, clothing, and shelter. It is the lived experience of commuting, doing chores, cooking, and eating. It accounts for adequate rest and sleep, maintaining one’s health, childcare, relationships, and community. It is the equivalent to the time spent surviving in nature—foraging for food, searching for shelter, but set in a modern milieu—shopping, chores, and conducting economic transactions. Reproductive time is always subject to the dominant time of its age, be that irreversible time, chronological time, or spectacular time. The economy considers reproductive time as a secondary concern. As such, health, aging, and childcare are continually sidelined by the economy as externalities that have no bearing on the production of products and services. Reproductive time is presented as the realm of the domestic, as feminine, and as such is considered invisible and intuitive—which is to say, lacking in thought and rigor.²³ Reproductive time is, in fact, the opposite— To quote Marx describing the revolutionary proletarian: ‘I am nothing and I should be everything.’ (Marx 1975, p. 254).it produces the visible workforce, and a lot of emotion and thought is put into care for the people who become the fundamental basis of the economy.²³ According to Angela Garbes: Garbes 2022, p. 14.Love and care, like social change, are slow and follow circuitous paths that take days not hours, years not months. The work may seem inefficient, but love doesn’t play by the same rules as the economy.²³ While work is perceived as ‘apart from life’, and ‘free time’ remains perceived as ‘more authentically real life’, the lived experience of reproductive time is cut off from ‘real life’, remaining undocumented and unaccounted for on any personal narrative. This deprives daily life of language and as a formal concept, and it lacks any analysis of what events occurred within its own past. Reproductive time is the real time of life, but one that is misunderstood, smothered, and forgotten—to the benefit of the false consciousness of spectacular time that is the memory of the unmemorable.³ Reproductive time is the primary site of recuperation by neoliberal capitalism as it continues to colonize this time. It appears across the globe with the defunding of any social services that support reproductive time. Concrete examples of this include the defunding of public education and the increase of private schools and private childcare; the increase in the costs of college education, the politicization-of & elimination-of reproductive healthcare; the increase in health insurance costs and its decreasing coverage; the reduction of subsidized food and housing programs; the privatization of health services such as the NHS in the UK, and the consolidation of hospitals and outpatient clinics in the US. These reductions come at the expense of workers, and the costs are borne by workers in terms of both time and money. The ideological goal of these actions is twofold: shift the burden of reproductive time onto workers by eliminating services and shifting them to the private domain while corporations capture increasing profits by decreasing operational costs.

I chose to use Debordian term ‘Lived time’ because it has the connotation of actually living. Other have used the Greek word ‘kairos’ to describe such moments, see Odell 2023.Lived time is the freest, most fun, most creative mode of time experienced.³ Lived time is the Kairos, Debord 2004, p. 62.those strategic points in which we are obliged to sieze the moment and act,²⁴ those nonlinear and qualitative moments that dynamically stretch and contract with our emotions, engage all our senses, and requires full participation.²² Lived time are those Known as ‘ecstatic time’ in Being and Time, the word comes from the Greek ékstasis—to be ‘outside of oneself’. See Heidegger 1962.moments of ecstasy experienced during leisure, play, focus, intense emotions, and erotic rapture.⁴ It is a time experienced without the awareness of time passing. Lived time are those moments when we do something for the hell of it, escaping the rigid order of the schedule. Lived time is a The ‘space of play’ is Graeber’s term for a distinction between play and games, where play is free, creative, and open-ended, while games are rigid, repetitive, and closed-off. Graeber suggested that play underlies art, science, conversation, and community, while games are the preferred method of bureaucracy. He emphasized that the key distinction between games and play is the presence of rules, with games being defined by their rules, while play is defined by its lack of rules (Graeber 2015).space of play that is free, creative, and open-ended.²⁵ Athletes, artists, and creatives have converged on describing this time as “a state of flow”—a total state of immersion in activity. Lived time is unhurried, a time in which people can pursue their own projects, at their own pace. These projects may not lead to a predetermined goal, but gradually become unto themselves, absent a goal—done for their own sake. When we say we “have lost track of time”, it is often due to the experience of lived time. The feeling of lived time at its most intense can be depersonalizing, we only realize how absorbed we are when we emerge from it. Lived time is never scheduled and always late. It is here where we abandon the utility-value of time as money, and it is where past, present and future coalesce into duration²¹—the perception of memorable times distilled into a narrative that chronicles one’s life. Lived time is fundamentally useless, ephemeral, disposable, and therefore cherished—that singular characteristic beyond value. Lived time is experienced for its own sake—not guided by instrumentality or utility. It is a time that uniquely gives us a sense of our own mortality.²¹

Instrumentarianism and The Perpetual Present

The Perpetual Present is our contemporary social milieu—a period without duration that never changes, it is a temporality where the past is irrelevant, and The Perpetual Present is the reification of time itself.the present moment is spent determining a future that isn’t our own. Unlike the circle in the cyclical time of nature, and the line in the irreversible time of history, the perpetual present is a point of singularity collapsed into a time we call ‘now’. The perpetual present is the period at the End of History.

The The autonomous economy is an economy which has shifted from the satisfaction of human needs to the generation of pseudo-needs in for its own sake, external to any human desire, or even society itself, organizing reality in order to reproduce itself.autonomous economy privileges the perpetual present which is concentrated as a cascade of urgent ‘nows’ that bury the past and obscure the future. There is no felt duration between past and future when the now is in ubiquitous crisis. The ubiquity of this temporality creates a dominant social order—one in which we scurry to meet the demands of our quantitative culture.²¹ Our lives increasingly exist in the perpetual present, where moments are sold according to the ticking of the clock. Time is money, time is getting things done, and yet, at the end of the day we cannot account for where our time went. A society dominated by continual busyness is a social life that is permanently anxious and “It’s easier to imagine the end of the world than the end of capitalism” a quote attributed to Fredric Jameson and Slavoj Žižek (Fisher 2009).too distracted to imagine a future.²⁶²⁷ The perpetual present is haunted , according to Mark Fisher, by lost futures that the twentieth century taught us to anticipate… the disappearance of the future meant the deterioration of a whole mode of social imagination: the capacity to conceive of a world radically different from the one in which we currently live. It meant the acceptance of a situation in which culture would continue without really changing.²⁸ This is a society deeply networked by smartphones, and an onslaught of tasks that ruthlessly undermine any sense that time can ‘congeal’ due to informational saturation. The more information we are exposed to, the more we are subject to the durationlessness of the perpetual present, and our focus narrows onto specific instrumental objectives.²¹ This instrumental attention prioritizes time solely as a means to attain certain ends: we allocate time for tasks essential to perceived survival, tethering our actions to the ticking of the clock. This gives us a diminished sense of duration, and therefore diminished memory—a society of amnesiacs in pursuit of productivity for its own sake, in service to an economy that operates for its own sake. In the perpetual present we are all Leonard, the protagonist in the 2001 film Memento.

Cf. Nolan 2001Here’s the thing. You see, I haven’t told you about his condition.²⁹

Leonard has anterograde amnesia, a result of a violent home invasion that left him injured and his wife dead. The traumatic memory of this murder drives Leonard’s quest for revenge. Despite his condition, Leonard is portrayed as smart, determined, and capable. He is the exemplar of the achievement-subject: someone focused on achievement for its own sake, the task of increased achievement. Leonard exists in a perpetual present brought on by amnesia. Spoilers: spoiler alert in three… — two … — one … — once he has gotten his revenge, he must answer for himself: What is next, and to what end? To this, he refuses to answer, and instead changes his own memory by editing the notes that document his own past. His perpetual present is constructed with lies and selective memory. He continues his task beyond completion, beyond achievement. Leonard is the achievement-subject taken to its logical conclusion.

Our perception of the perpetual present exhibits two key characteristics. Firstly, time is exclusively tied to the fleeting and present moment. Secondly, this present moment becomes illusory, infinitely divisible, and laden with an ever-expanding array of tasks—becoming infinite. Pursuing this logic leads to the imposition of an unattainable burden, resulting in widespread anxiety and burnout.¹⁹

Leonard’s amnesia inoculates him from burnout. Without memory, without any meaningful sense of the past, Leonard maintains an infinite state of productivity without burnout. The TV show Severance expands upon this idea, where employees volunteer to have their brains surgically segmented, thus splitting one’s work life from their personal life—they, quite literally, have no memory of their workday. In the show, The procedure is legitimated as beneficial to mental health, and its’ effect produces a docile, burnout-free workforce.³⁰

We are all Leonard in the perpetual present that is continually demanding ‘more’. More things to do, more tasks to complete, more labor to be expended, more psychological effort, more status to achieve—nothing is ever complete, and nothing is ever enough.²¹ We, unlike Leonard, are not immune to anxiety, depression, exhaustion, and burnout. Leonard exemplifies instrumental attention, an attendance to goals and outcomes that is typical of the achievement-subject. These goals and behaviors are—much like Leonard—vulnerable to manipulation by the operant forces of Spectacle and surveillance capitalism.

Zuboff defines surveillance capitalism as the utilization of big data, AI, and data-driven technologies to subtly modify human behavior remotely and at scale towards economic ends.³¹ The smartphone inaugurated the era of big-data surveillance and gamified behavioral modification using gadgets. Twitter’s “pull-to-refresh” UI gesture that leverages variable rewards, much like a slot machine, triggers addictive behavior as a way to increase user engagement with the timeline. Facebook employs similar psychological methods with the “red dot” notifications that keep people checking their phone for the next new thing.³² Youtube & Netflix discovered that auto-playing the related videos dramatically increased views.³³ Zuboff defines According to Lacan³⁴, the ‘big Other’ represents a kind of ideal observer or societal norms that individuals perform for, shaping their behavior and expressions to conform to social expectations (Evans 1996). Žižek³⁵ further explores this concept by delving into the enigmatic nature of the ‘big Other’, questioning its demands and highlighting its role in shaping desires and behaviors within society (Žižek 1989).’Big Other’ as the global physical architecture of computerized and networked technologies that achieve a pervasive and unprecedented means of behavioral surveillance. This creates a new form of Instrumentarian power that operates through the manipulation of individuals’ behaviors, preferences, and actions based on predictive data derived from technological tools—ultimately serving the commercial interests of corporations.³¹

Contrary to Orwellian depictions of oppressive surveillance technology that maliciously watches our every move, instrumentarian power doesn’t rely on forced control or subjugation but rather on individuals who willingly interact with technology in ways that lead them to progressively disclose personal information and behaviors over time. Zuboff’s Instrumentarian power aligns with Debord’s concept of the Integrated Spectacle in Comments on the Society of the Spectacle he described it as a fusion of the Diffuse and Concentrated forms, characterized by intensive technological development, continual personal re-skilling, and an integration of state and economy that mirrors how surveillance capitalism intertwines technology with economic interests to monitor and influence individuals’ voluntary experiences for profit. It is a prescient description of instrumentarian power (Debord 1998, ch. IV). Voluntary participation is both far more effective and efficient at subduing the masses, since it appears to empower individuals through improved productivity and consumer choice.¹⁴ Technology companies leverage cloud computation, big data, artificial intelligence, and behavioral economics to monitor and manipulate experiences for the purpose of commodifying users’ private data and attention to keep users engaged.³¹ Ultimately, instrumentarianism is a power over time, a unilateral claim to the future,³¹ a power to sell future behaviors to advertisers—all while burying the present in a cascade of banal trivialities.

The Rise of the Zeitzombie

The extreme busyness of the perpetual present saps us of vitality and leaves us without a strong sense of personal identity. We have been seduced into the culture of hustle & grind, we have worked so many long hours that our imagination is devoid of anything outside of work.

”There is a sort of dead-alive hackneyed people about, who are scarcely conscious of living except in the exercise of some conventional occupation.”³⁶

said Robert Louis Stevenson. Accordinng to Han, Han 2015, p. 50.When life is devoid of value, it must be kept healthy at any cost, health becomes self-referential without any greater purpose. Health is the new goodness in a society that doesn’t know what good is… The people of the achievement society are zombies, they are too healthy to die, and yet too dead to live.¹⁹ This is all to say, We are all Leonard, we are—the zeitzombie.

Zeitzombie zeit·zom·bie, IPA /ˈzaɪtˌzɑmbi/ A conjunction of the German zeit for ‘time’ and the Kikongo word zumbi for ‘fetish’. The zeitzombie A form of commodity fetishism applied to time: as “the confused identification of particular objects [time] with the attributes that are accorded to them as a result of the social relations in which they are located”. Bunyard 2018, p. 296.confuses their time with money. The zeitzombie is a living corpse reanimated by productivity and busyness. They are those who claim to have no time. The zeitzombie is created when a life is drained by the vampire of neoliberal capitalism, converting living flesh into Dead labor is labor power that has been expended into a product, a widget, a machine, or even a factory (Marx 1867).dead labor.³⁷ Symptoms include: the glorification of work, a protestant work ethic, and a proclivity to do things for the sake of getting them done, and dedication to keeping healthy at any cost. Other symptoms include a resistance to sociality, a fear of daydreaming, and the elimination of boredom.

The zeitzombie is the product of our collective proclivity to say ‘yes’, it’s an Positivity, according to Han, is “the societal focus on positivity and achievement can lead to an excess of pressure and expectations, making it difficult for individuals to manage negative experiences and ultimately resulting in burnout.” (Han 2015, p. 1). Cf. Fisher 2014 who calls it a “permanently multitasking, permanently distracted state of being”.excess of positivity.¹⁹²⁷ As the achievement-subject, we say ‘yes’ to the urgency of the perpetual present, we say ‘yes’ to performance improvements at work, we say ‘yes’ to optimizing our health, we say ‘yes’ to answering late night work requests, we say ‘yes’ and pickup those side jobs. According to Jenny Odell: Odell 2019, p. 15.Every waking moment has become the time in which we make our living… time becomes an economic resource that we can no longer justify spending on ‘nothing’.³⁸

There is no ‘work-life balance’ for the zeitzombie, when all of life is work: producing our work, producing our bodies to better produce work, producing our psychologies to identify with the production of work. According to Fisher: Fisher 2009, p. 34. Work and life become inseparable. Capital follows you when you dream. Time ceases to be linear, becomes chaotic, broken down into punctiform divisions… To function effectively as a component of just-in-time production you must develop a capacity to respond to unforeseen events, you must learn to live in conditions of total instability, or ‘precarity’.²⁶

There is a type of zeitzombie that exists merely to solve ‘real world problems’—the techno-optimist. This species goes as far to claim that even death can be fixed with AI³⁹ to quote Andreessen —presupposing death as a problem in need of a solution, and yet offering no vision for what life could be. Give us a real world problem, and we can invent technology that will solve it.³⁹ they say. Everything is a problem to be solved, and technology is the solution to every problem, and yet the zeitzombies propose no reason other than for the sake of inventing new technology. They offer no vision for a good social life—only a world in which Emphasis in the original. For the techno-optimists, work is heaven, and in their manifesto there is no meaning, nor life, in their section called “The Meaning of Life”, cf. Andreessen 2023.man was meant to be useful, to be productive.³⁹ , in their words. For the zeitzombie, there is no life outside of work, and death is a problem that needs to be solved in order to guarantee future productivity. The zeitzombie is the techno-zealot— quoting him again, we believe intrinsic motivations– the satisfaction of building something new…, the achievement of becoming a better version of oneself, these are the words of the achievement-subject, true believers in the coming of the Their techno-Messiah is the General Artificial Intelligence (AGI). “We believe Artificial Intelligence is our alchemy… [AI] can save lives” (Andreessen 2023).AI techno-Messiah⁴⁰ who will unleash endless productivity for its own self-development, or Roko’s Basilisk is a thought experiment that proposes a superintelligent AGI who punishes anyone who doesn’t actively contribute their lives to the development of itself. (Roko, 2022)punish those who refuse to work for its own benefit. And so, just like the knights in the Age of Kings whose personal chronicle was bound by duty to their lord, the techno-optimists are bound by duty to accelerate The Singularity: the emergence of their AI god.

The zeitzombie has internalized corporate claims that Fisher 2009, p. 40.‘satisfactory’ is not enough, they intuitively understand that schools, jobs, and even gig apps will undertake performance improvement plans unless they’ve met ‘exceeded’ or ‘excelled’ performance. They are therefore compelled to excel beyond satisfactory in all institutional environments. This drive to continually excel and constant precarity leads to a diminished sense of life and yields to the simple concern for survival. This is the ‘logic of increase’ that subsumes all notions of a good life in the realms of work, money, health, knowledge, relationships, and community. This logic manifests as a fear of falling down the social order that is amplified by social media as we scroll through friends’ and celebrities curated image of “what life could be”.²² The condition of the zeitzombie applies equally up and down the social spectrum, from the working-poor to the Elon Musks of the world, everyone is time-poor when we all live to work.

With the conveniences offered by smartphones, we willingly forfeit the separation that work and life once had. These technologies expand the legitimate claims to our time inside and outside work, we are expected to be Odell 2023, p. 65.reachable by anyone anyplace and at any time.²² The work-life balance is blurred even more when you consider the impact of technology on leisure. Vacation once meant leaving the physical place of work behind, now we must ask ourselves what is vacation when you can work remotely from vacation? When there is no clear demarcation between work and where we are in space, leisure, as it was understood, is now less definable—just as the concept of ‘home’ is less obvious when we work from home. That which was formerly private space is no longer a place of refuge.

Depression, Attention Deficit Hyperactivity Disorder (ADHD), Borderline Personality Disorder, and burnout are the dominant psychological pathologies at the beginning of the twenty-first century.¹⁹ The zeitzombie is ultimately a victim of a society that privatizes social problems, Fisher 2015, p. 21.treating them as if they were caused only by chemical imbalances in the individual’s neurology and/or by their family background—any question of social systemic causation is ruled out.²⁶ says Fisher. Society puts the responsibility of managing social problems onto the individual as personal failures. Failure to socialize, failure to manage time, failure to manage one’s own mental health—and yet depression is the , according to Han: Han 2015, p. 10.pathological symptom of late modern human beings failure to become himself.¹⁹

Thus, as we forfeit our claim to time, and forfeit our claim to space, we forfeit our claim to memory, we forfeit our claim to life—and thus we all become the zeitzombie.

Leisure as Refusal: The Power of ‘No’

Over the last 40 years corresponds to the neoliberal era of economic deregulation that started with Lee Kuan Yew in Singapore, then Reagan in the US and Thatcher in the UK; Yasuhiro Nakasone in Japan, and Deng Xiaoping in China pursuing ‘Socialism with Chinese characteristics’.40 years we’ve seen an accelerated emergence of implicit and explicit forms of negativity and refusal, especially in more concentrated societies like Japan and China. The The Japanese economic miracle of the 1980s was characterized by easy credit, unbridled speculation, and soaring asset prices, which eventually led to a bust in the early 1990s, resulting in a deep recession.Japanese economic miracle and the subsequent stagflation of the 1990s created a challenging environment for young people, especially university and high school students, who faced immense pressure to succeed in a society where work achievements were hamstrung by loyalty and seniority. The intense competition and high-pressure environment led some individuals to withdraw from society as a coping mechanism, resulting in the condition of the The term ‘hikikomori’ was first used in 1985 to describe retreat neurosis and social apathy, it originated in Japan and refers to a form of social withdrawal where individuals, typically young adults, isolate themselves from society for extended periods, often staying at home and avoiding social interactions (0xADADA 2003).引きこもり.(hikikomori)⁴¹ The The 996 working hour system (996工作制) in China requires employees to work from 9:00 am to 9:00 pm, 12 hours per day for 6 days per week; i.e. 72 hours per week. It is still widespread as of 2024.996 working hour system in China resulted in exhaustion, burnout, mental health conditions, and strained personal relationships among workers. This led to the emergence of the 躺lyingtǎng平flatpíng movement in the spring of 2021. Tang ping reflects a growing discontent among Chinese youth with the intense work expectations and lifestyle sacrifices demanded by the dominant work culture. Tang ping encourages individuals to ‘lie flat’, rotate the world 90 degrees, and people will discover this unspoken truth: the one who lies flat is standing, and the one who stands is crawling.⁴² Tang ping advocated for a recognition of the slavery inherent to existing working conditions, and rejects the societal expectations tied to relentless work hours and material success. Tang ping and its spiritual successor 摆let itbǎi烂rotlàn echoes the American hippie counterculture movement of the late 1960s, and it is the contemporary analog to the Quiet quitting phenomenon in the US. Quiet quitting gained traction during the COVID-19 pandemic, where employees fulfill their job requirements but do not go above and beyond, rejecting the idea that work should consume their lives and choosing not to exceed the expectations outlined in their job descriptions. They’re setting boundaries, avoiding overtime, and focusing on completing tasks within designated working hours. Taken together, these conditions are a conscious or unconscious withdrawl from society in what Berardi calls ‘desertion’.⁴³

To resist the dominance of spectacular time and the perpetual present, we must collectively take responsibility to determine how we use time & space—and to what ends, otherwise the instrumentarian power will allocate our lives to its own ends. To do so, we can try embracing negativity. By negativity we don’t mean melancholy or pessimism, but rather our ability to self-determine one’s life by negation, or the change of what we are with a view to being or becoming something else. Without negativity we do not have freedom.¹⁹ Neoliberal capitalism encourages human beings to be defined by activity and action—‘yes I can’ (an excess of positivity), often from pressures outside onesself. Negativity is the opposite, it is the power of ‘No’—the power to choose.

The power of ‘No’ is predicated upon the increases of productivity in the last 40 years without a corresponding increase in wages or a decrease in time spent working. American households are working more overall today compared to 1980, putting in an average of nearly 4 more weeks of work annually. This is largely a result of women having shifted from reproductive labor to employment.⁴⁴ This increase in productive-time yields surpluses in the form of profits for corporations, and yet lost time for ourselves. According to Marx, Marx 1973, p. 628. C.f. Debord 2021 thesis 1.The wealth of societies in which the capitalist mode of production prevails presents itself as an immense accumulation of commodities.⁴⁵ The real measure of wealth in a society that calls itself ‘developed’ is not money nor commodities, but rather, disposable time—which is to say the Bunyard 2018, p. 220.collective potential to actualize qualitatively differentiated moments of lived experience.² We say ‘No’ and refuse to give more time to the production and consumption of commodities.

When we think of time in terms of hours, minutes, and seconds, we mistake these mere referents as the thing in itself—which is to say, we mistake the map for the territory. The result of this perception is a collection of social problems caused by Odell 2023, p. 120, cf. Bergson 1944conceiving of the true nature of time as stemming from wanting to imagine discrete moments sitting side by side in space.²²⁴⁶ said Bergson. This thinking traps us into the anxiety of ‘losing time’, of ‘time passing’, and the loss of productivity that motivates the achievement-subject to produce their own self-exploitation. Many of our contemporary social conditions come from auto-disciplining ourselves into an inability to say ‘No’.

The power of ‘No’ allows us to move towards temporal self-determinacy. This is temporal negativity, a ‘No’ to things that hijack our time, a ‘No’ to instrumentarian power. The task of ‘No’ is to consciously (and collectively) self-determine the goal of human activity and act towards the realization of that goal. The individual’s concrete activity in leisure (the means) is directly connected to its own action (the ends), which is to say, it is done for its own sake. This is the fulfillment of unity between theory and praxis that moves beyond the economic determinism of the autonomous economy to our own, communal, self actualization in time and history. A path forward may be to extend the collective understanding of leisure as a way to establish Bunyard 2018, p. 211.a condition of unity between conscious human agents (subject) and their own objective historical existence (object), which is to say—we can become who we want, as we want, in our own time.

‟Rather, her [Sarah Connor] knowledge of the material conditions that produce this awful future have radicalized her, she has embraced her situation not as a passive observer, but as an agent active within history, fully conscious of the power that the present has over the future.”⁴⁷

What can revitalize us, is thinking. Thinking takes time that requires lingering, deliberating, weighing up possible alternatives, and patience. All these are active forms of ‘No’ that resist attempts to turn us into passive beings that obey without resistance. If we don’t, we face hyperactivity and its consequences: anxiety, depression, exhaustion, and burnout. ‘Hyperactivity’ is reactionary, it blindly reacts with unthinking efficiency—it lacks in activity since it presupposes immediacy, rather than real activity of thought, which requires time, effort, and delay.²¹ We can borrow practices from Zen meditation, where one attempts to achieve Han 2015, p. 23.the pure negativity of not to—that is, the void—by freeing oneself from rushing, from intrusive thoughts.¹⁹ We need contemplation: the mode of Merrifield 2008, p. 165.holistic thinking that considers everything, the thinking that is the product of daydreaming, adventures in the mind, meditative thinking that is often useless but done for its own sake.⁴⁸ We need activity of thought—the non-time of thinking without rules, without best practices, using intuition which brings into existence new territories of ideas, language, and actions that could never have been predicted.

To say ‘No’ is to reject the dominant temporality and embrace time for its own sake, a time that is ours.

What can ‘save’ us is the revival of the sabbatical, a period of relaxation and observation. The sabbatical, according to the Hebrew Bible, occurs after six years of planting and tending one’s fields, you let them lie For more on fallowing, see Kuzmanović 2024.fallow⁴⁹ on the seventh year—to let them grow wild without harvest or tending. It is a year of rewilding of the land, just as we may similarly re-wild our thoughts. Farmers took sabbatical on a rotating basis, and other farmers pooled resources to support the farmer who was observing sabbatical. The sabbatical is a period of listening, not speaking; watching, not doing. It’s not about refreshing oneself to improve future productivity, or reflecting on the past. It’s about reorienting oneself to the present moment free from outside demands.

What can ‘save’ us is idleness. Idleness and imagination are essential to our wellbeing, they’re the wellspring of mental clarity, and the backstop for memory. Idleness is the space of play, it opens a time that engages all the senses and full participation. It allows for activity for its own sake, without ends or instrumental reasoning. The activity is self-determined and open-ended.

What can ‘save’ us is boredom. Boredom is the peak of mental relaxation¹⁹ and acts as a gravity well for contemplation. We can turn off our smartphones and de-network ourselves. We can resist the mechanization of the supernormal stimuli, we can resist our reactionary impulses that drive social media by means of technological abstinence, it could be Orthodox and traditionally observant Jews do not use lights, electrical appliances, or any other electrical devices on the Sabbath (Fri night to Sat night). This is because using electricity violates the biblical prohibition on performing “creative labor” (melachah) on the Sabbath.ceasing to use technological devices, or it could be by creating spaces that are free from commercial technology.

What can ‘save’ us is leisure. True leisure is the experience of free and undetermined Lived time. By free we mean freedom, a mode of living that is not determined, modes of life free from necessity and purpose.⁵⁰ This is a leisure that is the opposite of vacation, which became instrumentalized leisure, sold as a packaged product to refresh us for our next stint of work. Leisure provides Odell 2023, p. 106.a moment of perspective on the grander reality²² where we can engage with both past and future. Through leisure we reanimate our sense of duration by reintroducing lived time into our lives. Leisure is the source of happiness, real meaningful happiness comes from the whimsical, where we engage in practices and projects which are not subordinate to an economic imperative (those things which are trivial, superfluous, and gratuitous). Put simply, leisure is to luxuriate in fun.

We demand a leisure of the commune, a leisure that isn’t the personal responsibility of the individual, but a leisure that is a common good for all people. This is leisure-as-public-park: where we Odell 2023, p. 95.have demarcated space as one free from private ownership… free from the work of optimizing oneself.²²

What can save us is to Income inequality is to such a degree that no amount of working reduces ones poverty. So our question remains, why work? “the same question is about to be posed again everwhere: how can we make the poor work, when illusion has disappointed and when force has been defeated? (Debord 2021, Preface to the Third French Edition)Never Work!³

1. Travel Writing - Redux, Italy & Switzerland Edition
2. Against Facebook
3. I am not a software developer
4. The Disappearance of Lived Time

Thanks to Jenna, Stephen, and Remi for reviewing early drafts of this essay.

Bibliography

Christopher Nolan’s 2020 sci-fi action thriller Tenet is a heist film in which the titular organization attempts to stop the end of the world through time travel.¹

Due to the sensitivity of the main characters mission, they use a call-and-response passphrase: “We live in a twilight world.” and “There are no friends at dusk.” This phrase is used to identify those on the inside of the mission from outsiders, and, thereby, avoid divulging secrets to an enemy that can manipulate time.

This passphrase is a reference to Hegel:

“When philosophy paints its grey in grey, a shape of life has grown, and it cannot be rejuvenated, but only reorganized, but the grey in grey of philosophy; the owl of Minerva begins its flight only with the onset of dusk.”²

An alternative translation reads:

“When philosophy paints its grey in grey, one form of life has become old, and by means of grey it cannot be rejuvenated, but only known. The owl of Minerva takes its flight only when the shades of night are gathering.”³

Hegel’s metaphor of the owl of Minerva signifies that true philosophical insight and wisdom only emerge at the end of an era or after events have transpired. The owl of Minerva spreads its wings only with the falling of twilight, symbolizing that philosophical reflection occurs when a way of life is mature enough to be understood in hindsight. This concept implies that genuine understanding and reflection on history and society can only happen once an epoch has concluded, emphasizing the retrospective nature of philosophical comprehension.

Commenting on Hegel, Papaïoannou describes his post-revolutionary philosophical maturity as a Papaïoannou 2012, P. 91’twilight world’⁴, as an era that could be interpreted, with equal justice, both as end of the day and as a new dawn. His metaphor thus indicates that philosophy need not contemplate a completed world, but could instead herald its transformation.

With respect to the meta-narrative of the film, it could simply mean “There are no friends at the end of the story” because (spoiler) Neil dies.

I’ll ask Mr. Nolan if he has read both Hegel and the commentary by Papaïoannou, and if the reference is correct.

From Q4 2022 through Q1 2024 the tech industry has seen a shit-ton of layoffs. These are due to a combination of factors:

1. The end of the ZIRP: Zero Interest Rate PolicyZIRP era, higher interest rates means investors demand profitability (instead of growth) from portfolio companies.
2. In 2017, Section 174 of the US tax code changed, taking effect in 2022¹.
3. Companies with a remote-first culture are hiring nationally, even globally².
4. A market correction from over-hiring during the pandemic.
5. The hype surrounding AI tools lead to preemptive downsizing³.

The unfortunate outcome of all these layoffs is the impact on these employees. Employment is a critical part of people’s lives, and layoffs can leave long-lasting emotional scars. A job is a zeitgeber: German for time giverzeitgeber, and aside from the circadian rhythm, it is the dominant structural organizer for the cadence of daily life. This is why unemployment is so destabilizing, it arrives as the feeling of vertigo—the ground below has suddenly fallen away, and that nothing is what it was. It manifests as a gradual sense of derealization is detachment from ones surroundingsderealization, as if the world isn’t real, as if you are in a dream. Unemployment is an anxious mental liminal space, a waiting room where control is largely in the hands of others—waiting for replies, waiting for call-backs, waiting for upcoming interview dates, waiting to hear back, wondering why. These feelings will pass, it is temporary.

The next hurdle is the mental investment in perpetual studying and the preparation for technical interviews (which are a poor predictor of performance). The labor is never good enough, never prepares you for all eventualities, and is never complete. Then there is the emotional labor of courting potential corporate suitors, imagining yourself in the role, asking the good questions, showing passion and craftsmanship only to hear them say: unfortunately we’re moving forward with other candidates.

You are not your job title. Being laid off says nothing about you or your worth, certainly nothing about your value as a person; they are simply the result of a calculation to further the interests of the company. Layoffs are a failure(unaccountable) of management that socialize losses(workers get laid off) and privatize profits(execs get paid off).

My Experience with Layoffs

For many this will be the first layoff experience of their career, and I feel for them. I’ve been here before, quite a few times in fact, perhaps you can benefit from my experience.

I started my career in software engineering as an intern at Rovia was an early e-book pioneera tech startup during the dot-com crash of 2001. My internship promptly ended when the company closed and laid off most of its employees the week after I started. Aside from not having a job to help pay for rent, I didn’t earn those course credits that were a part of my graduation requirement. I went back to university for six months and then had an opportunity to make up that course credit at EMC² (Now Dell EMC) the following year. That internship also ended with another mass layoff—this time because of the post-9/11 stock market downturn of 2002. For the remainder of my time at university, I focused on the course work. I let time sort out the economy while I wasn’t participating in it.

I tend to think of employment in tech as riding a bull. Not a big bull-riding spectator, but from what little I have seen, it really isn’t about whether you are going to stay on the bull or not. You are going to get thrown off, it’s just a matter of when.

Eventually I graduated into the post-crash era, and had a few good jobs. In a turn of luck, I managed to avoid the 2007–2008 financial crisis unscathed—despite joining the volatile startup n2N Commerce in September of 2007. The vibe there was off, and executive leadership was at odds with product leadership. I left n2N right before the December holidays after only 3 months, landing a job with a friend at interactive agency RDVO. On January 2nd, my former boss at n2N called me—n2N had just done a massive round of layoffs and was ceasing operations, he wanted to know if I knew it was going to happen. I didnʼt, but the vibe had raised my suspicions. I walked away unscathed by the holiday disaster, it would be a formative experience in trusting my gut.

I worked handful of other jobs before joining startup Cinch Financial. There we used behavioral economics to nudge people towards positive personal financial decision making. The the company eschewed recommending “pay-to-play” products to our customers: as a fiduciary, we actually did recommend the best products for our customers. It felt right. The team was experimental and fun. We had small and agile teams with a pragmatically groomed backlog. We ran tightly iterative dev-UX release cycles with continuous deployment. We encouraged a non-burnout culture. We tried things, failed, and had psychological safety to take real risks. Cinch ceased operations in September 2018 during an acquisition attempt when it simply ran out of runway and failed to take flight. The entire team was laid off. I co-organized the Boston Ember.js Community meetup while at Cinch, and that was a major contributor to helping me land up at Salsify—they were a frequent host of our meetups.

I joined Salsify in October of 2018, less than a month after Cinch. Like many tech startups of the ZIRP era, it was on a growth trajectory. It had a strong engineering-lead culture, some interesting R&D, a steady march of new product features, and we did a few acquisitions. Salsify went through 3 rounds of layoffs in April 2020, November 2022, and October 2023—the last of which eliminated the entire business vertical I was in, and many amazing people were let go, world-class engineers, designers, product & sales people.

Gareth Mallory: I only have one question. Why not-stay dead? James Bond: Do you get out in the field much? Gareth Mallory: You don’t need to be an operative to see the obvious. It’s a young man’s game.

After the layoff at Salsify, I quickly dipped my toe back into interviewing, if just to practice. I probably started too soon, I wasn’t really sure what I was looking for, however, it was good to practice interviewing.

2023 was a difficult year for my family, it was a year of crisis. My mother had a stroke, we had two relatives in the family die, our grandma went into the hospital, and our dog died suddenly (the latter four all happened between December 24 and January 2nd).

I finished out Q4 2023 by reflecting on my career and tried (and failed) to rest during the holidays. It was my season of wintering. Katherine May describes a wintering as a fallow period in life when you’re cut off from the world, feeling rejected, side-lined, blocked from progress, or cast into the role of an out-sider. Perhaps it results from an illness or a life event such as a bereavement or the birth of a child; perhaps it comes from a humiliation or failure. Perhaps you’re in a period of transition and have temporarily fallen between two worlds. Some winterings creep upon us more slowly, accompanying the protracted death of a relationship, the gradual ratcheting up of caring responsibilities as our parents age, the drip-drip-drip of lost confidence. Some are appallingly sudden, like discovering one day that your skills are considered obsolete, the company you worked for has gone bankrupt…⁴. A few of those felt especially relevant to me, namely “caring responsibilities as our parents age”, the “drip-drip-drip of lost confidence”, and the fear that “one day that your skills are considered obsolete”—especially in the face of the impostor syndrome resulting from interviewing and the looming threat of AI.

My wintering provided some space to daydream, reflect, and complete projects I’d been too preoccupied to do:

• reflected on my time at Salsify
• connected with friends.
• installed a new kitchen faucet.
• replaced old laptop batteries.
• consolidated a sprawling mess of cloud services to iCloud.
• read a few books.
• pruned & caught up on a large backlog of tech blogs.
• took some online courses in newer web tech.
• reorganized bookshelves.
• digitized our family photos.
• finished a bunch of personal projects.
• 🥳 refactored my Neovim config to Lua & LSP 🎉.
• refactored this blog.
• started writing on topics that I’d been Woolgathering refers to indulging in aimless thought, daydreaming, or contemplation without a specific purpose. It can involve allowing one’s mind to wander freely, fostering creativity and introspectionwoolgathering for far too long.

My Job Search

I re-started the job search in earnest in early January and shifted my search to mature, publicly traded software companies. I’m convinced that startups in the US are in for a very difficult time unless theyʼre in AI, crypto, or tough tech ventures address significant real-world problems, often with long development timelines and high technical uncertaintytough tech.

After a few months of searching, it is hands-down the most difficult job market I’ve ever seen. There is a pool of world-class engineers available. I personally know more great unemployed talent now than ever in my entire career. I suspect the Department of Labor statistics arenʼt reflecting how bad it really is because many of these workers are havenʼt filed for unemployment because a privilege, but one every worker should gettheyʼre still on severance. The interview process is far more difficult both in terms of technical breadth, depth, ‘soft-skillsʼ, and incidentals. Employers are expecting “batteries included” engineers who require 0-training, 0-investment, and fully-fit to their needs. Itʼs an employers market, so they expect to get the best, most committed engineers, and they’re able to down-level at a discount.

For the first time in my career, I got no responses to applications, or companies ghosted me after the initial call. privilege of the software engineerNeither of these had happened in any previous job search. If it is this difficult for a cis, white, male startup veteran, what are the chances for a non-white, non-male, with less experience and without a network to fall back on? In the past, job interviews for senior roles had fewer rounds, but now they’ve increased to 7-8 rounds with more code challenges. There are now at least 2, often 3, coding exercises, along with discussions about technical experience, product & design, leadership, and behavior. The difficulty level of coding questions has also risen from LeetCode easy to medium/hard. As the process progresses, companies become more selective, especially since they have many strong candidates. The final decision often depends on smaller details, what particular library or framework you’ve used, or simple “match making.”

Some take-aways: I didn’t make it through the interview process for any startups, only the established companies considered me. I’m not sure why, maybe startups mimic big companies’ processes without proper validation, or perhaps it’s age-related—I’m considered a greybeard now. Their technical questions tended to be easier. I had one interviewer ask me “how many dependents will you have on your healthcare plan” which I pointed out is an illegal question. He stumbled back with “Oh, I ask because our individual plan is good, but our family plan isn’t good”—he went on to describe their company culture as “we hustle, we’re usually in the office 6-days a week and long nights”. They are co-opting the hustle culture ethos and filtering for those rise-and-grind 20-something ‘entrepreneurs’. Is self-exploitation cool again? I know burnout isn’t. I noped out of that funnel, they went into the ‘Declined’ bucket along with another company that pushed back on the value of contemplative thinking and only valued calculative thinking⁵.

My job search, by the numbers:

• 🗓️ 125 Days Unemployed
• 📆 83 Workdays (6 major holidays)
• ➌ 3 Interviews/Week
• 🗣️ 52 Interview Rounds
• 📝 20 Applications
• 📞 13 Phone Interviews
• 🚫 9 Rejections
• 👻 7 No Answers
• 🧑‍💻 8 Coding Interviews
• 🧑‍💻 6 Pair Programming
• 🧑‍💻 6 Technical Case Studies
• 🧑‍🔬 6 Systems Designs
• 🧑‍🎨 4 Product Experience
• 📈 3 Leadership Experience
• 🧑‍⚖️ 3 Management Interviews
• 🛑 1 Position eliminated pre-offer
• 🏆 1 Offer Accepted

Advice Column

I’ll end with some advice to those still searching for work. Reach out to ex-coworkers and ask for LinkedIn references, endorsements. Reconnect with people, in real life. It’ll be good for the soul, and you can provide moral support and references for each other. Tending to your network creates luck. Ask yourself whats next. If you’ve never thought about where you’d like to be in three years, go for a walk and think about it. If you find it hard to wake up excited about work, ask yourself why, and give yourself space to think about it. Ask yourself what skills you want to learn, and start learning them by doing a project. Don’t learn for learning sake, but apply it to a project, even if an artificial one. Reflect upon your time at your last company. Write a narrative of your time you can use during interviews to describe your experience. Focus on your milestones, challenges, learnings, mistakes, growth and accomplishments. Write a long-form career narrative answering the “Tell me about yourself” question. Write it for yourself, but then shorten it to a 1-2 minute summary that you can recite during interviews. Refine it with use.

There is a ton of great talent out there, and even more impostor syndrome. I fear that a lot of great talent will start to pivot their careers away from the tech industry. Just remember, everyone is getting rejections, don’t take it personally. You are the main character, but layoffs arenʼt about you: they’re about make number go up 📈.

If I had written this article in my optimistic twenties or thirties, I wouldʼve ended with a rousing call to action: “Knowledge Workers Unite!”, but I’ve been around enough to see those efforts spark & fizzle⁶, and spark & fizzle again⁷. Iʼd advocate that we should still vote against Trump, but Bidens policy also supports the neoliberalism & globalization that is at the core of these boom-bust cycles and the cause of the flattening of our wages across the globe. Unemployment is a full time job, and it takes immense effort to keep interviewing and maintain enough energy to continue personal projects. The obvious path forward for workers in a globalized economy with increasing underemployment is Universal Basic Income.

In lieu of UBI, my call to action is something more intimate and karmic: be gentle with yourself, reconnect with yourself and those around you. Rediscover yourself. Work on your own projects, and help others with theirs. Through contemplation and connection—magical things can happen.

Almost a year of working in Ember, I’d like to summarize my thoughts about what makes it a great framework. There are a million articles comparing the big JS frameworks; this isn’t one of those.

Almost eight years of working in Ember, I’d like to revisit some of the things I said about Ember. There are a million articles comparing the big JS frameworks, this one is mine.

Ember vs React

React allows us to create very declarative code, but to do it idiomatically, the way React wants takes quite a bit of experience and refinement to do it the right way. When examining purely declarative code, especially to the uninitiated, it can be hard to comprehend when states are responding to events, and what their downstream effects are. React code can be very composable, and their useEffect, useCallback, useState primitives can be combined to create a wide variety of functionality from these granular affordances.

With the composability of hooks, however, comes a level of complexity. We spend a lot of time thinking through how different code is going to run, when it was going to run, and how it could potentially interact with other hooks and code around it. It all makes sense in the end, and it’s also declarative, but it’s a bit harder to follow the intent all the way through.

React (for most of us) is usually only legible in motion, not at rest. This is to say that the only way to figure out how a given piece of React code works is by prodding it through use, it’s not enough to just read it.

Differences in developer feeling of focus Reacts dogmatic re-running of a component when its state changes forces developers to consider all starting and possible states, and their dependencies. It requires juggling a lot of context on the part of the developer, and that is why reducing state to the minimum is considered a best practice. By contrast, Embers autotracking makes component development feel a lot less complicated, and an Ember developer can focus more on intent while React developer feels they must focus more on exactitude. These are just feelings, but they’re there.

Ember is more baroque with its @action and @tracked decorators, its template control logic with {{#if}} {{#each list of |item|}} and its native classes. With this baroque-ness however, comes an additional level of legibility one gets from reading Ember code. Ember code feels familiar everywhere, and separating arguments(props) from attributes really helped to clarify intent.

Ember suffers from a small and shrinking ecosystem. This could be due to early technical decisions that required all 3rd-party Ember npm packages to specifically adhere to Ember addon requirements. This meant that regular npm packages couldn’t be used in Ember apps until ember-auto-import arrived, and later, Embroider. Unfortunately this means most Ember addons aren’t compatible with the wider javascript ecosystem, and many Ember packages languish or are abandoned once maintainers move away from Ember. This means the Ember core-team is burdened with adopting these orphaned packages or risk losing them to the shifting sands of time.

Direct Comparison

When speaking to Ember devs about React, or React devs about Ember, I frequently use this chart to map concepts in one to the other:

Eight Years On

Revisiting my original post, I’d like to provide some updates to that thinking:

Ember feels familiar…

Ember still feels familiar, especially for teams that work across many Ember repos. Ember teams are very productive because of Embers convention-over-configuration approach. An Ember app is an Ember app is an Ember app. This familiarity isn’t always typical with React apps. React apps are to-each-their-own, and teams need to spend time to make their apps feel familiar enough to be seamless when moving between repos.

There is a single “Ember way” of doing things…

This isn’t as true as it used to be. Ember is wonderfully backwards compatible, but that also puts it into an awkward position when it comes to writing idiomatic Ember applications: It has become very difficult to discern what is idiomatic Ember, especially in real-world app ecosystems where you may have apps using Ember (classic) or Ember Octane, or an in-between state. Ongoing projects like template imports will make Ember apps even more difficult to determine what is “idiomatic”.

At Salsify, there are a handful of Ember-gurus who shepherd the engineering org towards idiomatic Ember, but the average engineer doesn’t necessarily know when:

• Ember.get/set vs. native getters/setters
• Component.extend({}) vs class Component extends Component {}
• invoke component with {{my-component}} or <MyComponent />
• render a value with {{value}} or {{this.value}} or {{@value}}
• when to use @tracked or @action

The Ember cheat sheet is a great resource, but its there because people aren’t confidant. Tools like eslint, prettier, stylelint and others help migrating to idiomatic conventions.

Concluding

Ember and React are both great. There are tradeoffs on both sides. The answer is always it depends. React with class Components is very similar to Ember Classic. Ember is an amazing upleveler for bigger teams with novice and expert developers working across multiple codebases building ambitious applications. React is elegant in its simplicity, can be pure and precise. React has an amazing ecosystem, and is used to create enterprise applications. Ember has a larger API surface area, and while it seems to have a higher learning curve, I’d argue that it takes longer for an engineer to become an expert at React: one having the ability to see the forest for the trees.

The panning shot is a technique where the camera rotates horizontally to capture a wider scene, is a tool that Villeneuve wields masterfully. However, it is his marriage of this technique with the parallax effect that truly elevates his storytelling. The parallax effect, achieved by creating the illusion of depth through the relative motion of foreground and background elements, draws the audience’s gaze and involvement into the frame. Villeneuve’s application of this combination immerses viewers in a tangible sense of space.

In “Blade Runner 2049”, Villeneuve employs this technique to stunning effect. The sprawling cityscape of a futuristic Los Angeles is captured through sweeping panning shots that showcase expansive slums, towering skyscrapers, and soaring advertisement holograms. The parallax effect comes into play as the foreground elements, like the protagonist’s vehicle, move independently of the background, emphasizing the vastness and intricacy of this dystopian world. The audience is enveloped in a reality that feels both immense and tactile, rendering the futuristic setting almost tangible.

”Enemy” takes a more intimate approach to this technique. Villeneuve uses the panning shot and parallax effect to delve into the psychological landscape of the protagonist, creating an unsettling atmosphere. As the camera pans across the stark, brutalist apartment buildings, the subtle movements of buildings in the foreground against the background hint at the duality and disorientation that plagues the protagonist’s mind. The technique plays on the audience’s subconscious, luring them into the unsettling mindset of the characters and enhancing the film’s enigmatic nature.

In “Dune”, Villeneuve’s mastery of the panning shot with the parallax effect reaches (literally) new heights. The vast desert landscapes of the planet Arrakis are captured with breathtaking beauty. As the camera moves horizontally, the towering sand dunes and rocky formations seem to stretch endlessly, amplified by the parallax effect as the foreground Ornithopters shift subtly against the rocky landscape. This technique transports the audience to the arid world, making them feel the expanse and danger of the environment as if they were walking alongside the characters.

By leveraging the parallax effect’s ability to provide a tangible sense of depth and space, Villeneuve offers an unparalleled engagement with his films’ environments. This not only showcases his prowess as a director but also emphasizes the potential of cinematic techniques to evoke emotions and connect audiences with the stories they behold.

In the words of Guy Debord, the creator of the concept of the “Society of the Spectacle,” Oscar embodies a rejection of the false realities and empty spectacles of modern society. He lives outside the system, literally on the fringes of society, and does not participate in the relentless pursuit of material possessions and status.

Oscar’s embrace of dumpster diving and his apparent lack of concern for hygiene and cleanliness can be seen as a rejection of the wastefulness and excess of capitalist society. In Marxist terms, the garbage that Oscar collects and lives among can be seen as the detritus of surplus value, the discarded byproducts of a system that prioritizes profit over all else.

As a critic of wider society, Oscar occupies a unique position. Living on the streets, he is literally on the front lines of climate change and other environmental issues. His rejection of mainstream values and way of life makes him a powerful symbol of resistance to the destructive forces of capitalism and consumerism.

This essay was co-authored with ChatGPT using the following prompt

Write an essay placing Oscar the Grouch within the Hellenistic philosophical tradition, comparing him to a contemporary Diogenes the Cynic. Describe Oscar the Grouch as if you were describing Guy Debord, and make references to The Society of the Spectacle. Elaborate on how Oscar the Grouch rejects consumerist society, how he embraces dumpster diving and the wastefulness of capitalism. Define the term ‘garbage’ as the the detritus of surplus value using the Marxist definition of surplus value. Describe his position as critic of wider society, focusing on ho he literally lives on the front lines of climate change.

The art for this essay was co-created with MidJourney using the following prompt

Oscar the grouch as Diogenes the cynic in the style of a white marble statue

In 2020 I wrote about the potential of the Subaru BRZ but its lack of a track-oriented trim level:

If we want a truely lightweight track-oriented weapon like the 2002 WRX STI Spec-C Type RA, we’ll need to build it ourselves… or really, deconstruct the cars we have, in order to make one.

… so I built one myself.

It had been five years since I had owned a track car, it was the early months of the COVID-19 pandemic, I was languishing, and my wife said “You need to do something, you need to make your own happiness”.

Your Own Happiness

‟Your Own Happiness” is the name I’ve given to a 2017 Subaru BRZ purchased in October 2020 with 66k miles. The build mission is a trackable street car with a whole bunch of very subtle OEM+ aesthetic changes.

I’ve given myself a few principals to guide the build:

• The Subaru BRZ / Toyota 86 is a perfectly balanced drivers car for those on a budget.
• This build will be a trackable street car, in that order.
• Power is a band-aid for poor setup and lack of driving skill.
• Nobody is impressed if you can only pass in the straights.
• Weight and simplicity are key: Complexity means more maintenance, more weight, more cost, more heat, and more futzing at the track: reducing fun and seat time.
• A car that reinvigorates the idea of driving delight at an affordable price.
• Drive it to the track & back
• Make power with 93 Octane

After a six-month search for a used Crystal White Pearl kouki (2017-2020) BRZ or 86, I found one on the Subaru Certified Pre-Owned web portal in Annapolis Subaru. I called them and immediately put down a deposit. I rented a car and drove 11 hours on a Friday night, starting at 10pm, arriving at 9am Saturday morning. The car was in great condition, and lucky for me, this was just before COVID-19 caused used-car prices to skyrocket.

Many, if not most of the performance improvement ideas were taken from my previous RASpec Impreza build, and applied to this car, only simpler.

Thus far the car has seen 18 track days between racing & drifting, and has handled them beautifully, with only a few minor issues. It’s not making ridiculous amounts of power for a street car, keeping reliability and affordability in check. Most importantly, it’s fulfilling its destiny as a driver’s car, customised for its intended use and used well.

The car itself, the physical object—isn’t what makes me happy, its the involvement in creating my vision with my own hands, it’s the participation in motorsports, and of course the people and friends I make along the way. That is my own happiness. What is yours?

Engine

Tomei Japan EXPREME Ti equal-length manifold, Fujitsubo Authorize R TypeS cat back exhaust, ECUTek tune (188hp/145.6tq) by R Tuned at Kinetic Motorsports, HKS oil cooler, Odyssey PC1100 battery, Beat Rush sound generator delete, Perrin fuel rail brackets.

Drivetrain

Cusco engine mounts, Cusco steering rack bushings, Cusco shift lever bushings, Perrin rear shifter bushing, MTech shifter springs, STI transmission mount, Cusco transmission mount collar, Cusco braided steel clutch hose, Verus Engineering forged clutch fork & pivot, BattleGarage RS throwout bearing, PBM rear subframe bushings, Cusco rear differential bushings, Cusco 1.5-way LSD, Cusco differential cover.

Suspension/Brakes

Öhlins Road & Track coilovers, OEM knuckles modified for more steering angle, SKP (extended) tie rods, Hotchkiss front endlinks, H&R 15mm spacers, Cusco front strut bar w/master cylinder stopper, DBA 5000-series 2-piece front rotors, Spiegler braided steel brake lines, CounterSpace CSG Spec C1 & C11 pads, Ferodo DS2500 brake pads, ARP extended wheel studs, Muteki SR48 open-ended lug nuts.

Wheels/Tires

ENKEI RPF-1 17x9+35 in Matte Black on Bridgestone POTENZA RE-71RS in 225/45R17.

Enkei GTC02 in Hyper Silver F 17x7.5+38 R 17x9+43 on Goodyear Eagle Exhilarate F 215/45/R17 R 225/45R17.

WedsSports TC-105X EJ Titan 17x9+35 square on Yokohama ADVAN A052 225/45R17.

Rays Engineering VOLK Racing TE-37 Saga SL 17x9.5+45 on Michelin Pilot Sport 4S in 245/40/R17.

Exterior

Subaru STI bumper lip and lip skirt, CLEiB carbon fiber side skirts, Cusco tow strap, Verus Engineering carbon fiber side markers, Aerocatch hood pins, Vermont Sportscar Carbon Fiber hood vents & gurney flaps, Quik-Latch front bumper release, Audi Q3 overfenders, Perrin license plate bracket, Subaru ‘Type RA’ badge (from WRX STI).

Interior

Bride Zeta IV seat, MOMO Type 78 steering wheel on Works Bell quick release with steering controls relocation switch kit.

Verus Engineering throttle pedal spacer, P3 v3 ODB gauge, Safety Restore Ferrari-red custom seatbelt webbing, JDM Toyota/Subaru OEM interior trim & door handles, Amerex & Element fire extinguishers.

Gallery

Having our identities bound to our work, and most dangerously, to an employer severely limits ones ability to negotiate wages when that identity is immediately laid bare in monetary value—our human value is our wage.

Beyond wages, this mindset limits the horizons of what is possible to do with our own lived time.

”I need to do software engineering forever, for I am a software engineer”. We can create art with software, we can also create software as art, we can also choose to not make software, and we are still the same person. Software is something we create; we also create ourselves.

As someone who has spent a lifetime in software engineering, who has credentials in the field, and who’s hobbies and interests have, at various times, fallen into the bin of “software engineering”, I have often identified my worth with my current job, wage, and employer. I have put all my eggs in one basket. Identifying who we are with what we do is a cognitive limit.

This is to say, I refuse to self-identify only as invested in my work, my career, my achievements. This might have been okay where we talked about the idea of a career as a vocation, but not now—when the only horizon of intelligibility available according to the dominant mindset is one where we understand our selves only in terms of pure utility, or economic productiveness. This I refuse.

I do software engineering. I am not a software engineer.

1. Travel Writing - Redux, Italy & Switzerland Edition
2. Against Facebook
3. I am not a software developer
4. The Disappearance of Lived Time

In this book, Debord describes and critiques the way we live. The power of these ideas lies in their ability to question, identify, and name the common assumptions of the present. Debord develops the concept of The Spectacle, which describes the gaze of contemporary society. From its publication just before the May 1968 revolt in Paris, and ultimately influencing Occupy Wall Street, this book continues to transform a wide range of progressive philosophical and political movements, most notably anti-capitalism, postmodernism, marxism, and anarchism.

Debord’s work is not an ivory-tower philosophical treatise, it is a cold analysis of the history and development that leads directly to our present moment. He critiques various attempts to change society by comparing their advantages and disadvantages. Ultimately, he makes it apparent that a way out is absurd, and points to only one path forward: a direct democratic movement of decentralized workers councils, a suggestion made only twice in the entire text. Most importantly, he emphasizes the importance of validating theory with practice, and as such, this book is a practical framework for revolutionaries who think and do.

Debord’s text is concise, economical, poetic, provocative, and difficult. As Michael Hardt & Antonio Negri describe it in their notes to Empire as Quoting Hardt & Negri: “…Debord recognized this spectacle as the destiny of triumphant capitalism. Despite their important differences, such authors offer us real anticipations of the path of capitalist development.” Continuting in their notes: “[The Spectacle], which is perhaps the best articulation, in its own delirious way, of the contemporary consciousness of the triumph of capital.”delirious.¹ His work contains many references to Hegel and Marx. He uses words and phrases defined in other Situationist texts, and expects the reader to be familiar with them. This is to be expected, as he demands much from the reader, and had written for a small audience of “fifty or sixty people”² who were well versed in the history of the western philosophical tradition.

Debord was concerned that these ideas themselves would be recuperated by capitalism (see thesis 203) so he wrote in a way to limit them to a small and curious audience. His concerns turned out to be warranted, when a few years later in the early 1970s, the French Socialist Party under François Mitterrand co-opted the Situationist phrase “Change Life” as his campaign slogan, and Situationism became the party’s unofficial ideology—much to the consternation of Debord. His ideas were difficult because they were abstract, but the abstraction meant that it continues to remain relevant, arguably even more relevant half a century later.

The very first time I read The Society of the Spectacle, I knew I’d need to re-read it, and possibly need to re-write it in my own words if I wanted to truly understand it. It was only once I had grasped the gist of the text, that I was then confronted with the task of understanding the philosophy behind it. Debord didn’t lay out his ideas using plain language, because his ideas are austere, terrifying, and extremely dangerous. He is the kind of philosopher whose ideas and observations of the world are so bleak that one fears these ideas reaching mass consciousness.

This book isn’t as much a straight translation as it is an elaboration, or ‘remix’ that attempts to reveal more under the text than existed in the original and its various translations. Debord hid some ideas. This is an attempt to contextualize and reveal abstract ideas by bringing in references and annotations in order to add relevance for our current time. I have added a few thoughts and examples to help elucidate difficult concepts. In the hope of making Debord’s work more accessible to those unfamiliar with Marx, Hegel, and the breadth of other works and concepts alluded to in his text, I’ve knowingly ossified my own translation. The reader is strongly encouraged to wander from these pages back to the timelessness of the original—and beyond.

Foreign languages are much like distant places, and the act of translation is to visit these places. We may recognize similarities, but these places are unique because they have distinct histories. With languages, words not only have semantic meaning, but attachments to the rich cultural narratives that tell their histories, and these words are attached to libraries of other texts within the same cultural milieu. When translating these words, the histories, narratives, and cultural aspects most relevant to the original text aren’t always translatable with words alone; words simply aren’t enough to capture the rich cultural depth that exists between the words. As such, I’ve tried to add notes where the text made implicit references. When translating, “plagiarism is necessary”, it demands embracing the author’s ideas, and making them semantically and culturally relevant, and if done well, it can create an entirely new work, a copy without an original.

This edition isn’t merely a translation of words, but a translation of time. Since Debord wrote this book, society hasn’t changed in kind, but by degree, everything is exactly the same, only more. Advertising is no longer prominent, but dominant. Facebook and Google aren’t merely the tech companies driving the economy, but have captured the entire advertising industry, monopolized it, and built the foundations of a new form of capital on the back of data tracking, profiling, and machine learning: surveillance capitalism. Urban development has increased separation and inequality to public goods like education and transportation. Technology is now ubiquitous and we’re all glued to personalized screens all day, mediating nearly all interactions, even the ways we find love. For Debord, none of this would be new, only more.

The book before you is my attempt to participate in the development of his ideas. I’ve tried to ground some of his more abstract ideas upon the terrain of the recent economic development of surveillance capitalism. The emergence of commercial social media, particularly, is the confluence of a few concepts described by Debord: celebrity culture, the perpetual present, and mass media that work to automate the commodification of personal identity backed by machine learning.

If you’re looking for a translation that aims to stay faithful to Debord’s classical French prose, this book is not that. There are editions perfectly suited for that purpose, I would direct you to the translations by Ken Knabb, Donald Nicholson-Smith, or Fredy Perlman—all of which I referenced extensively during the preparation of this book. I’d especially like to point out how valuable Ken Knabb’s annotated translation has been as a resource for this edition. His work handed me most of these references, and was instrumental in pointing me in the right direction for a few others. If you are looking for new insight and academic rigor, I would refer you to Russell, E., 2021 and Bunyard, T., 2018. All of these editions can be found in the bibliography and are well worth reading.

I’ve arranged the notes along the margins rather than as endnotes because they’re meant to be read alongside the main text. If you are reading this book for the first time, I would suggest reading it in a particular order, as the first few chapters can be discouraging. I agree with the preface to Ken Knabb’s 2014 translation, in which he suggests starting with chapter 4 and 5 because they provide relevant historical and revolutionary background that helps to contextualize the book. From there, read chapter 7 which covers the development of cities, urban development and social issues. Then move to chapter 8 which covers culture, the arts, and the history of artistic movements. Finally, read chapters 1, 2, 3, 6, and 9 which establishes his concept of The Spectacle and provides a comprehensive critique of contemporary society. Thus, my suggested chapter reading order is: 4, 5, 7, 8, then 1, 2, 3, 6, 9.

My hope is to make Debord’s ideas more accessible to first time readers and to show how much more relevant The Spectacle is today than when it was first written. I am optimistic that together we can make another world possible.

-ra March 2021

Black Mirror: Bandersnatch (2018) is an interactive film in which viewers are periodically asked to make decisions for the main character Stefan Butler, the decisions then result in different scenes, and paths through the film, resulting in a branching path to a variety of endings. Think “choose-your-own-adventure”, but in interactive video.

Since television viewers are conditioned to be passive spectators, the film included a tutorial early on to help viewers become familiar with the choice mechanic. The first example asks the viewer to make a seemingly trivial choice of which breakfast cereal the main character Stefan eats for breakfast: Kellogs Frosted Flakes™, or Quaker Sugar Puffs™.

by giving the viewer the power to change the course of the storyline, the viewer’s role changes from one of voyeurism to one of active surveillance. Nevertheless, and in reality, the viewer’s new power only increases the tracking and profiling of his/her choices. We, the viewers, might control Stefan, but Netflix (the producer of this commodity) panoptically monitors and surveys us¹.

The choice mechanic not only shows the viewer how choices are presented during the film, but demonstrates how the choice is stored and recalled by the film later when a retro commercial with Tony the Tiger plays on a TV in the background: programmatic product placement. This simple mechanic has wide reaching implications for a future of personalized content marketing.

This mechanic can be infinitely scaled, it would be easy to replace the cereal box with a simple green screen, and inject the ad in post-production. In fact, similar technology is already being used by broadcasters of sports matches, where they show different advertisers on the perimeter boards based on the viewing market. The difference, of course, is these ads would be placed per-viewer rather than per-market.

The first phase of advertising was display advertising, brands created single non-personalized campaigns, to appeal to the widest possible audience. This campaign included billboards, newspaper & magazine ads, television, and radio advertisements. The ads were either stand-alone with billboards, or content-adjacent with magazine, television and radio advertisements. Placement was determined by non-specific general demographic data.

With the arrival of the Internet, advertising shifted to online advertising in which real time bidding platforms placed ads that were tailored to appeal to certain demographics. This method allows for more control for the advertiser (or agency) over the individual target audience, rather than just the website or app it appears on.

With advancements in social media and machine learning, behavioral advertising emerged as a method to place highly relevant, personalized ads in relevant user contexts that increases influence in brand-positive consumer outcomes. In effect, behavioral advertising collects data from web browsing behavior and app engagement, finding patterns that lead to purchasing outcomes. If a consumer was frequently searching for plane ticket prices, the targeting system would recognize this and start showing related adverts across unrelated websites, such as airfare deals on Facebook. Its advantage is that it can target individual’s interests, rather than target groups of people whose interests may vary. These ads can be personalized and unique. These ads are content-adjacent.

Where Bandersnatch Marketing is different is how it integrates the technical advancements of machine learning and big data from behavioral advertising and marries advertisements directly into the content. In effect, these ads are not content-adjacent, but content itself.

With this approach, the cereal Stefan eats wouldn’t necessarily have been an obtrusive viewer choice, but instead based upon a real-time ad placement chosen by an advertising networks knowledge of the individual viewers behaviors on the Netflix platform or from data collected from other ad networks. For a young kid it could’ve been a bowl of Lucky Charms™, for the health-conscious adult siggi’s yogurt, and the Paleo-eating CrossFitter it could’ve been an Epic™ meat snack bar.

What this means is, if done well, the ads disappear and become content, seamlessly. The ads no longer influence buying behavior as much as the entire experience gently nudges the viewer, more subversively, and more efficiently. The success of this approach lies in subtlety, the product placement must not be frequent nor blatant.

Even the role we, the interactive viewers, play has now changed. Instead of being passive viewers/consumers, Netflix has given us the opportunity to become the new gods, an omniscient presence that controls the Black Mirror world. A closer look, however, shatters this reality, for the more control we are given, the more we are manipulated.

Superficially, this approach solves a problem for Netflix: How do you make money advertising to a userbase who’ve become accustomed to paying for an entirely ad-free experience? The approach of content-oriented product placement seems reasonable, and Netflix will be able to associate products with content, microgenres, or specific demographics. Additionally, it will be able to directly test product designs (e.g., two different Frosted Flakes box covers), as a service Netflix could sell to brands before production begins.

But the bigger business Netflix is building is the underlying infrastructure to support content-oriented experiences through interactive storytelling and personalized product placement.

The next phase of innovation points beyond personalized choice of content to personalized content in which the media itself is customized, tailored, or generated unique to each individual—leaving them unable to relate to one another with respect to their media experiences, shared culture, and ultimately we lose some of our shared perspective on the world around us.

Just like Google built the backbone of search advertising, Netflix is building the backbone for a future of video storytelling where the ads and the content will be impossible to distinguish because they’ll become the same thing.

This is the OEM badge that appeared on the 2018 Subaru WRX Type RA edition, of which only 500 were made. The STI’s Type RA suffix stands for Record Attempt, in reference to the highly modified WRX STI Type RA NBR that Subaru used to set a lap record at the Nürburgring Nordschleifes for a sedan at 6”57s. This was also the very same vehicle Mark Higgins drove at the Isle of Man TT course and set the record at 17”35s. Japanese manufacturers love badging of this type, and famously the Nissan Skyline team badged their GT-R R34 with a “V-Spec II Nur” emblem to mark their achievements at the Ring.

The “Type RA” in Subaru parlance means the raw form, usually lighter more than anything else. The production 2018 STI type RA got 5 more horsepower, slightly shorter 3rd gear ratio, inverted Bilstein front struts, carbon fiber roof and rear wing and 19” wheels with 245/35R19s and put the car on a tiny diet losing only 53lbs off the regular 2018 STI, which is still a beast at nearly 3,380lbs. For all that reduction, they still add nearly $12,000 to the price tag, pumping the monster up to the $50k pricetag. We’ve truly entered the age of postmodernism when the marketing team advertises “reduced” features and actually adds to the pricetag. Fat-free, gluten-free, and free-range, but you’ll pay extra for it.

I am more interested in referencing the 2002 WRX STi Spec-C Type RA, which stood for “Rally Applicant”. This car was stripped of all luxuries: sound system, air con, power windows & locks, trunk trim, airbags, underbody coating all in the name of weight reduction. It was 116lbs lighter than a base model WRX. It had a cool roof vent and a quicker steering ratio. It had hand-ported heads that flowed far better (note: when building the engine for the RASpec Impreza, I found a set of Spec-C heads, and they were my favorite part of that build.) It produced more power and yet remained affordable because it was stripped down.

In 2020, legally, Subaru can’t sell a production vehicle without air bags, and supply chains dictate costs, which means that power windows are actually cheaper than roll-up windows because the entire product line uses the same motors. Part sharing at scale as large as Subaru means part sharing savings make it cheaper than deploying roll-up windows in a small-scale production run as well as keeping spare-parts on hand for repairs and service. Air conditioning has become a marketing necessity, as have power door locks. ABS brakes are also legally required to pass local safety standards along with standard shoulder seatbelts.

If we want a truely lightweight track-oriented weapon like the 2002 WRX STI Spec-C Type RA, we’ll need to build it ourselves… or really, deconstruct the cars we have, in order to make one.

On my car, the RA badge is a reference to that first 2002 WRX STi Spec-C Type RA, the purest and most raw form of the WRX. Oh, and I almost forgot, it stands for a reference to the driver, me, “Type Ron A.” 😏

Currently auto manufacturers must provide independent auto repair shops access to the same diagnostic and repair information about a vehicle as the manufacturers’ dealers have, but the current law doesn’t include telematics, defined by the state as “systems that collect and wirelessly transmit mechanical data to a remote server.” The ballot measure proposes increased accessibility to telematics via a mobile app, allowing vehicle owners and independent repair shops to read more vehicle information and run diagnostics.

As a Car Guy™, Right to Repair is a huge issue for me, personally. The manufacturers flat-out dont give any details on their engine managment units, they’re basically black boxes. So if i want to upgrade my engine, brakes, etc, i have to either hack the ECU, or find a 3rd party company who has hacked it. If the ECU was an open platform, then 3rd party companies could legitimately make verifiable upgrades. Some 3rd parties are super sketchy and make unsafe changes, while others are very well tested. Because there is no open access or transparency, its basically the wild west.

The other aspect of this, is that manufactuers can lock the car owner into buying parts only from the manufacturer, creating a monopoly on replacement parts. As cars become more electronic, this will become a bigger and bigger problem and deeper monopolies. As a race car builder, i’d love to see open platforms where people can created 3rd party parts and services that compete with the OEM manufacturers.

Case-in-point: after the 2008 economic crisis, many car companies disappeared (SAAB, Pontiac, Saturn, etc). Now, their cars weren’t heavily locked with electronic monopolies, but if they were owners would have no supplier for replacement parts because they’d be locked to a proprietary system that has shut down.

Open access and the right to repair means that 3rd party companies can always build replacement parts and alternative ECUs.

Finally, a big part of this is the misuse of the DMCA. Basically, if someone like me starts selling ECU “tunes” for my Subaru BRZ to other car enthusiasts, Subaru can sue me for “copyright infringement” for modifying the ECU of my own car and helping others modify theirs. This is about the right of the owner to repair, change and use their own vehicles as objects not services owned the the manufacturer.

The DMCA issue is particularly a problem in the automated tractor & farming industries. Companies like John Deere are now running “automated tractors as a Service” (ATaaS) and the farmers have to pay for it as a subscriptions service, and its hugely expensive. It means little-guy farmers can’t really afford it to compete with the big corporate farms. So the little farmers want to write their own software for John Deere tractors that is custom for their farm. The right to repair allows for that.

This is about people owning their personal property, and their ability to do with it as they see fit. This includes repairing, or changing these objects and data.

dscl . delete "${HOME}" JPEGPhoto
dscl . delete "${HOME}" Picture
sudo dscl . create "${HOME}" Picture "${PWD}/avatar.jpg"

Replace Time Machine with Volume on the menu bar

defaults write com.apple.systemuiserver menuExtras -array \
  "/System/Library/CoreServices/Menu Extras/AirPort.menu" \
  "/System/Library/CoreServices/Menu Extras/Volume.menu" \
  "/System/Library/CoreServices/Menu Extras/Battery.menu" \
  "/System/Library/CoreServices/Menu Extras/Clock.menu"

Show day of week, date time in 24H time in menu bar e.g. “13 Fri 16:49”

defaults write com.apple.menuextra.clock \
  DateFormat -string "d EEE HH:mm"

The United States was started as a rational experiment in democratic capitalism based on ideas that inspired its forefathers. They were influenced by ideas coming out of the Enlightenment, specifically Adam Smith, Locke, Rousseau, and the conception of humans resorting to “the state of nature” and acting in bad-faith with one another.

fuck the monarchy, let’s experiment with a rational government based on contract law and this new capitalism stuff.

The problem with this type of Hobbesian thinking is “man will resort to kill or be killed in the ‘state of nature’” and therefore the State must enforce contacts to ensure civility and protect the rights of property.

My thinking is more aligned with Zeynep Tufecki:

We think of society as this Hobbesian thing, as opposed to the reality where most people are very friendly, most people are prone to solidarity.

In disasters and daily life, people want to help each other, and feel as part of something greater than themselves. The atomization of the individual, a result of capitalism, continually undermines this fundamental aspect of the human experience.

So when we’ve built our government on Hobbes as our fundamental philosophical ideology, we’ve ended up institutionalizing the idea that civil society is “every man for himself” rather than “every man for each other” and those institutions become a self-fulfilling prophecy. Thus, we’ve expanded on the US experiment in capitalism by privileging corporations with more rights and freedoms than humans, but without the social or moral responsibilities. That can only end badly when their only motive is profit. Then we’ve built this enormous apparatus of violence in the US military, the national guard, a decentralized police force, and a word salad of agencies like the NSA, CIA, FBI, CBP, TSA, ATF, hell even the USPS has an armed force. All these agencies assume the average person is acting in bad-faith, and yet we wonder why these armed agents are killing people by the thousands and without accountability and most often with total immunity.

This is all to say, the experiment in democratic Hobbesian capitalism should be cancelled. Its core assumption are all wrong: it assumed infinite resources on a finite planet which is wrong, it assumed the worst of people which is wrong, It absolved corporations of any ethics and granted them freedom and rights—which is wrong.

The experiment is no longer producing the outcomes for which it once promised—based on the idea of the social contract established by Rousseau, which provides for social reciprocity: we chose the laws under which we live. This is less and less the case. We’re increasingly seeing wealth for the few and social precarity for the many, under a system of governance that responds to corporations before it responds to people.

It’s time to try a new experiment in resource allocation and governance that is egalitarian, sustainable, and provides a vision where all inhabitants of this planet act in good faith towards a project for multiple worlds, and many positive futures. Other worlds are possible.

The first steps to address the problem of systemic racism in Somerville is two-fold:

• Defund the police by at least 60% or more; and
• Hold the Mayor accountable to direct these funds to non-disciplinary social programs; and stop internal line-item transfers to the SPD.

The weapons and practices we use abroad always manage to find their way home. We’re seeing some of the torture and humiliations that occurred in Abu Ghraib here at home in ICE detention centers and local police stations as evidenced by the deaths of Sandra Bland and so many others.

Under to 1033 Program, military surplus such as guns, night-vision googles, surveillance equipment, teargas, drones, LRAD, and kevlar body armor all of which has been tested abroad is now sold or even given to our local police forces. This militarization has lead to the increase of police violence nationally, spanning a long history from the Watts Rebellion to Rodney King, the Kneel Movement and most recently with #BlackLivesMatter. This is yet another example of the cycle of racist imperialism coming home.

Dr. Casey Delehanty and colleagues at Gardner-Webb University found¹ that districts that received the maximum 1033 transfers are twice as likely to have police killings compared to districts that receive none.

There is little evidence that such militarization reduces crime, and considerable evidence that it both alienates communities and escalates police violence against their own population.

We’re also seeing the effects of imperialism on our soldiers coming home and finding the most relevant jobs in the police force, and yet their mandates are diametrically opposed. Many of these soldiers carry the mental scars of war, are being streamlined from the streets of Fallujah to the city blocks of the US with little mental heathcare or transitional training².

It is therefore in the best interests of our community to demilitarize and defund the police, and in doing so, we work towards a more just society that applies to all members of our community.

What is the police? They are the active guards of the commodity, those dead objects created by the labor of underpaid and exploited workers—the police have the job to ensure that the products of this exploited labor remains a commodity. These commodities have the magical property that these objects must be paid for, instead of remaining mere passive and inanimate objects (sneakers, fancy watches, iPhones & Teslas) that could be subject to anyone who comes along to make use of them.

In rejecting the humiliation of being subjected to the police, looters are at the same time rejecting the humiliation of being subjected to the media-celebration and glorification of these dead objects (most often produced during the exploitation of their own labor) into commodities.

Looting, in other terms, is both a rejection of the commodity and the violence used to protect its spectacular status, and a Peoples Bailout by other means.

References

Ember has three methods for invoking components and helpers in a template, either of the three can be used to invoke both classic and modern glimmer components.

The “classic” syntax in the form that uses curly braces {​{...}}, e.g.

{​{user-profile firstName="Dan" lastName="F."}}

The “angle bracket” syntax that uses a HTML-like form <... />, e.g.:

<UserProfile @firstName="Dan" @lastName="F." />

Finally the s-expression (for sub-expression) form used inside curly and angle bracket syntax to invoke a sub component or helper. This takes the form (concat a " " b), thus:

{​{user-profile name=(concat firstName " " lastName)}}

and

<UserProfile @classNames={​{concat "sticky " (if this.isActive "is-active")}} />

Angle Brackets

Similar to the vanilla DOM APIs that distinguish between JS properties from HTML attributes, angle bracket component invocations have two different namespaces you’re operating against.

You are probably most familiar with HTML attributes, which tell the browser how to draw an HTML element. These attributes can do things like defining the alt text on an image <img alt="A bird"> or the URL on an anchor tag <a href="https://example.com">. Angle bracket syntax implements attributes in a similar way, allowing the developer to apply these attributes to a DOM node somewhere in the component’s template.

<!-- parent.hbs -->
<UserProfile class="abc" @tagName="figure" />

The ...attributes syntax determines where the attributes passed into a component from an angle bracket invocation should appear in the component’s template. Any number of attributes and element modifiers can be specified on the user profile component now, and they will all be applied to the element that has ...attributes on it.

<!-- UserProfile.hbs -->
<figure>
  <img ...attributes src="default.jpg" />
</figure>

the resulting HTML output is:

<figure>
  <img class="abc" src="default.jpg" />
</figure>

Anything prefixed with @ is an “argument”, is passed to the component by its caller, is accessible to the class backing the component, and can be any JS runtime value. Unlike attributes, which tell the browser what to render, arguments tell your custom Ember component tag what to do.

<!-- parent.hbs -->
<UserProfile @name="Dan F." />

This template invokes the <UserProfile> component, which expects one argument: @name, the value we pass is the hardcoded string “Dan F.”.

In the backing component class, arguments are namespaced on the this.args object, which is immutable. You can access this argument as:

// user-profile.js
this.args.name; // "Dan F."

To illustrate the differences, here is an invocation of an angle bracket component using both arguments and attributes:

<Foo
  @bar="{​{123}}"
  @baz="{​{hash"
  a="1"
  b="hi"
  }}
  class="hello"
  data-fizz="ok"
/>

In the above, bar and baz would be arguments with the values 123 and { a: 1, b: 'hi' } respectively, while class and data-fizz would be attributes that could be applied to a DOM node somewhere in the component’s template.

Curly and s-expression Syntax

Ember curly (also called “classic”) component invocation only have one namespace: the argument, everything is treated as an argument, as if you implicitly included @ in front of each one. Historically, this was compensated for by automatically applying the value of the class argument to the class attribute of a component’s root element (if it had one), and this behavior could be extended to other attributes using the Ember classic attributeBindings API.

<!-- parent.hbs -->
{​{user-profile bar={​{123}} baz=(hash a=1 b='hi') class="hello"
data-fizz="ok"}}

In the above, bar, baz, class and data-fizz would all be treated as arguments to the receiving component, so any ...attributes in its template would be a no-op. In Ember classic components, all of these arguments can be accessed in the backing component class using either the argument name alone, <argument name>, or using this.<argument name>, thus:

// user-profile.js
bar; // 123
this.bar; // 123
baz; // { a: 1, b: 'hi' }
this.baz; // { a: 1, b: 'hi' }
this.class; // 'hello'
this["data-fizz"]; // 'ok'

Usage in the template is:

<!-- user-profile.hbs -->
{​{bar}}<br />
{​{this.bar}}<br />
{​{baz}}<br />
{​{this.baz}}<br />
{​{this.class}}<br />
{​{data-fizz}}<br />
{​{this.data-fizz}}

results in the following output:

123<br />
123<br />
[Object]<br />
[Object]<br />
hello<br />
ok<br />
ok

It’s important to note that both Ember classic and glimmer components can be invoked by either syntax: classic curly or angle brackets—and what matters is how the backing class is defined. Thus, if the above example was implemented as a modern glimmer component, the backing class must access the argument bar with this.args.bar. If the above example was implemented as a classic component, the backing class must access the argument with either bar or this.bar.

So What’s Missing?

For those following closely, you may have noticed that this leaves us with some kind of hole in the programming model, because there’s no angle-bracket equivalent for setting up a contextual component to pass in as an argument or yield out as a block parameter. That’s the topic under discussion at Ember RFC issue #497.

In the meantime, <div class={​{@class}} local-class="whatever" ...attributes> will ensure that a class applied to a component will pass through correctly regardless of how it’s being invoked.

A central issue of contestation is elasticity versus discipline, elasticity for me and discipline for you, alchemy of banking and money funding for me, austerity and borrowing/payment for you.

Like 2009, the (coming/arriving) financial collapse of 2020 will reveal the asymmetry of power between the financial sector and the rest of the economy, and it’ll especially highlight how the financial sector is continually privatizing profits and socializing losses.

When you see terms in the media about the Fed “expanding its balance sheet” think: “The Fed is literally buying failed financial instruments to prevent banks and financial companies from going out of business”. Currently, according to Section 14 of the Federal Reserve Act, the Fed is prevented from buying assets (stocks, index funds CDO: Collateralized Debt ObligationCDO, mortgage backed security directly on the market. Instead they can only buy government-backed assets, mostly bonds and MBSs by institutions like Freddie Mac that are government backed. Instead, the Feds first line of defense is lowering interest rates, and then creating liquidity by buying long-term maturities.

With the interest rates at zero, the biggest lever at the Fed is effectively broken, and now they’ve got to find another lever. March 6th, 2021Last Friday, Boston Fed President Eric Rosengren called on Congress to expand the Fed’s powers to buy and sell a broader range of assets. This will require congressional approval, and during this period of crisis, I’m certain Shock Doctrine politics will win out, and these ammendments will pass. I hope I’m wrong.

So what does this mean? This means that the Fed can start buying non-government backed securities: potentially any failing public stocks, private CDOs and MBSs¹. What this means is the Fed can buy the most dangerous, volatile, and toxic assets that nobody else is buying, the ones that are the first to fail in a bursting bubble. This is a Fed that has institutionalized & encouraged systemic moral hazard—saying, in effect: If you (the finance sector) create risky assets and they fail, we’ll allow you to profit during the good times and we’ll save you by buying them back during the bad times.

Remember the Nobody remembers the Zune, it failed horribly.Microsoft Zune mp3 player? Now imagine Microsoft had put so much investment into its Zune, that when nobody bought it, and the product failed, the company would’ve gone bankrupt. Now, if this were the financial sector, this is where the Fed becomes the Dealer of Last Resort, steps in and buys all the Zune mp3 players, saving Microsoft from going bankrupt.

This is exactly what the Fed is doing when it says it is “expanding its balance sheet”—its buying the failed assets the financial industry has created over the last 10 years, those very same assets the industry has made huge profits from; but now that the profits have stopped, the Fed bails them out by buying them when no one else will.

Update: January 2021Nearly 10 months² since this essay was published, we’re already seeing strong indicators that consumer purchasing power will be severely hurt by inflation. So while government stipends may help in the near short term, long term people will be hurt by less value for every dollar they earn.

When the Fed goes to congress to ask for changes to Section 14, they will use the Shock Doctrine playbook: the banks will fail, retirement funds will tank, companies will go out of business, we’re in desperate times, so we need to take desparate measures. These predictions are all true, and these outcomes will happen: but the solution they offer only addresses the symptoms, but not the root cause—the deregulation of the financial sector. Neoliberal deregulation of the financial sector over the last 40 years has encouraged banks to put money into risky bubbles that keep popping, retirement funds of regular people are directly tied into that system, even fueling it. This is wrong. I’m not generally opposed to these risky investments, but these investments must be separated from investment banks, and kept out of any fund designed for peoples retirements and long term savings. These asset types need to be drastically limited to qualified investors and speculators only.

As it stands, bailouts will reward the most risky investors and speculators, while passive investors, retirees, and taxpayers will end up bailing out these risk takers. We’ve already seen it happen, specifically, the gutting of the Volcker Rule, the repeal of the affiliation restrictions of the Glass–Steagall Act, and the giant free-for-all stock buybacks taking place.

The only ways to mitigate risk-loading is strict limitations on executive compensation, employee bonuses, and no mergers or acquisitions for at least 10 years. These banks should be nationalized, its execs fired and all contracts voided. New leadershp will be brought in as reasonably paid public servants.

Thus, for the finance sector, the Fed provides elasticity: fungible rules around banking, investing, taxation, and bailouts when things go bad. For the rest of the economy, and individuals, we get discipline: massive losses, austerity and cuts to social programs and public infrastructure, moralizing our individual failures as people, capitalizing on the large-scale failures of small businesses³, and continual indebtedness for the poor and middle classes.

This is an ideological conservatism combined with a blind faith in an economics that can never be wrong. As Frank Wilhoit put it Conservatism consists of exactly one proposition, to wit: There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect (See The travesty of liberalism by Frank Wilhoit 2018). Only the in-group is an autonomous, rogue financial sector, and the politial interest groups they fund. The out-group is everyone else.

In this way the finance industry can reap the profits of risky financial instruments, but when the risk generates market failures, the Fed bails them out. It’s all upside, and no downside. This is the very definition of moral hazard. The financial industry literally cannot lose because the Fed will always save them.

In effect, this is Socialism for the rich, rugged individualism for the poor —Martin Luther King, Jr. (pp 346).

Lemon socialism is a pejorative term for a form of government intervention in which government subsidies go to weak or failing firms (lemons; see Lemon law), with the effective result that the government (and thus the taxpayer) absorbs part or all of the recipient’s losses. The term derives from the conception that in socialism the government may nationalize a company’s profits while leaving the company to pay its own losses, while in lemon socialism the company is allowed to keep its profits but its losses are shifted to the taxpayer.

So the imbalance: Why does the Fed continually save the financial industry every 10 years, but not any other industry? Why does the Fed create and spend 1.5T dollars to purchase worthless and arcane financial instruments, but never spend on public goods like housing, student education, public infrastructure?

This is our biggest opportunity, while we have the most leverage, to heavily invest in public infrastructure and works programs. The airline industry is already asking for bailouts to the tune of $50B. This isn’t the time for bailouts, this is the time for imagination, to create a society that works for everyone. This is the time to nationalize the airline industry. Since 2010, the very same industry has been investing their money directly into stock buybacks, pumping up their own stock prices, and paying huge bonuses to their execs and CEOs. Since 2010 the major airlines have invested $45B in buybacks, and now asks for $50B in bailouts.

This means that they spent $45B of their own money betting that their stocks would go up in value, paying out any profits to their execs and CEOs as their valuations rose, and when the market burst, they asked for their money back and an additional $5B on top. To put this simply, this is a massive con-job and outright theft. Nationalizing the airline industry is a win-win: it will create good paying american jobs, make flying more affordable, mitigate cost cutting at the expense of flight safety (see ‘Boeing 737 Max’), and mitigate market risk.

And now we get to the systemic asymetries of power. Trump and those within his inner circle, as well as his political base believe they are entitled to the rights and protections of the law, and meanwhile cast blame and stigmatize those least able to defend themselves. Their in-group will be the first to receive the benefit of any bailouts, and the negative consequences will be blamed upon, and most adversely affect the out-group. As Adam Serwer puts it: the cruelty is the point:

Only the president and his allies, his supporters, and their anointed are entitled to the rights and protections of the law, and if necessary, immunity from it. The rest of us are entitled only to cruelty, by their whim. This is how the powerful have ever kept the powerless divided and in their place, and enriched themselves in the process.

We, as a society are starting to understand that the financial sector is rotten to the core, and this rot is going to seep into public sentiment: “If they don’t play by the rules, why should I?“. This has the effect or eroding the very conception of a nation operating under an equal rule of law, and will result in toxic resentment and fuel civil unrest.

This lil’ project is an animated SVG using CSS transforms to rotate the text.

The Laughing Man logo is an animated image of a smiling figure wearing a cap, with circling text quoting a line from Salinger’s novel The Catcher in the Rye, which reads:

“I thought what I’d do was, I’d pretend I was one of those deaf-mutes.”

This character deeply resonates with me; he/she is a corporate hacktivist infiltrating micromachine manufacturing corporations that hide an inexpensive cure to a debilitating disease in order to profit from their more expensive but proprietary micromachine treatment. They are able to hide their physical presence by editing themselves out of video feeds and surveillance systems in real-time by by superimposing the animated logo over their face.

The Laughing Man logo has been co-opted by pop culture in advocacy for the Electronic Frontier Foundation and the loose hacktivist collective Anonymous (using the latter’s motto We are Anonymous. We are Legion. We do not forgive. We do not forget. Expect us.

Code is up on github

$ yarn add @0xadada/random-emoji
$ node
> const random = require('@0xadada/random-emoji');
> random()
'😁'
let a = random();  // defaults to 'emoticons'
let b = random('emoticons');
let c = random('food');
let d = random('animals');
let e = random('expressions');
console.log(a, b, c, d, e);
// 😍 🙄 🍗 🐥 🤢

Small, fun.

Fascism is an expensive to maintain state-of-siege by the capitalist economy attempting to defend itself by an irrational means. Fascism rallies to the defense of the conservative bourgeois ideology of the family, private property, the moral order, and patriotic nationalism. It unites the petty-bourgeois, the unemployed who have been hurt by the crisis or disappointed by the impotence of the socialist revolution‒ it is not by its nature fundamentally ideological, but may exhibit ideology to suit its needs. Its strength is how it presents itself truthfully as a violent resurrection of mythic past origins‒ to claim the past as a “Golden Age” and a return to the success of this golden past. It demands participation in a community held together by mythical archaic pseudo-values: race, blood, and the leader.

Fascism is a cult of the archaic fitted out by modern technology. It revives and recreates its myth (both past and present) through the spectacle using speeches, television, radio, internet, and pop culture. It is a major factor in the formation of the modern spectacle. The false mythology of this “Golden Age” is one of the fundamental factors of contemporary society, and was a primary factor in the destruction of the workers movements of the past.

Fascism however, is the most costly means of preserving a capitalist order due to the scale and effort required of the state to enforce the authoritarian rule of a minority with a police state apparatus. Fascism is unsustainable in the long-term, and must be pushed aside by a more efficient and rational form of power; neoconservative or neoliberal representational democracy being typical contemporary examples.

uri-editor.js is 1-line of HTML that’ll run a text editor in your browser.

Drag the link into your bookmarks to save it as a quick browser-based editor tool for those moments you need a quick editor nearby.

#!uri-editor.js

I review traditional, single-page web applications, I discuss server-side rendering by introducing Ember FastBoot. I showcase our architecture and provide a quick summary of how we use FastBoot in production.

Using Ember FastBoot in Weird Ways (slides)

Recording

Presentation Slides

In this article I’d like to present a list of Ember addons that I use in most of my projects. I’ve been using Ember for the last few years as my go–to framework for developing web applications, and many of these addons make appearances in nearly all of them.

Ember addons generally fall into one (or more) category of functionality I’ll be referring to throughout this guide:

• Build-time Build-time addons provide command-line tools that help developers during the creation of the application. An example is ember-cli-eslint which provides code linting, or ember-cli-typescript which adds a build pipeline for transforming TypeScript files into JavaScript files. These addons don’t ship features to your deployed application. These addons typically start with the prefix ember-cli-.
• Runtime Runtime addons provide features that will be present in the final application, these include Ember components like ember-power-select and ember-svg-jar these addons increase the payload of the deployed application. These addons typically start with the prefix ember-
• Infrastructure Infrastructure addons provide features that aren’t shipped with the payload of your application, but provide functionality that improves the development ergonomics or deployment of the project. For example ember-cli-fastboot provides a backend Node.js server for rendering Ember apps serverside.
• Quality Assurance Quality assurance addons provide tools for improving quality of code over time, and improving the developer experience of writing and testing code. These addons typically provide functionality that is used at build and test time, but isn’t shipped to your deployed application. qunit-dom, coveralls, and ember-test-selectors are examples of quality assurance addons.

Some of these addons are included by default by ember new <project-name> but I’ll elaborate on their use a bit more.

Finally, before diving into the addon list, I won’t be discussing many standard JavaScript packages. There are a bunch of JavaScript packages that I often use (ramda, lodash, etc) but these are outside of the scope of this article.

Contents

• General Purpose Addons
  • ember-a11y-testing
  • ember-auto-import
  • ember-cli-update
  • ember-cli-code-coverage
  • ember-cli-dependency-lint
  • ember-cli-deprecation-workflow
  • ember-cli-document-title
  • ember-cli-dotenv
  • ember-cli-template-lint
  • ember-test-selectors
  • ember-truth-helpers
  • eslint-plugin-ember
  • eslint-plugin-prettier
  • prettier
  • qunit-dom
• Specific Usecase Addons
  • ember-cli-addon-docs
  • ember-cli-bundle-analyzer
  • ember-cli-deploy
  • ember-cli-mirage
  • ember-cli-page-object
  • ember-cli-release
  • ember-cli-typescript
  • ember-cli-fastboot
  • ember-cli-fastboot-testing
  • ember-concurrency
  • ember-css-modules
  • ember-intl
  • ember-intl-analyzer
  • ember-fetch
  • ember-power-select
  • ember-simple-auth
  • ember-svg-jar
• Conclusion

General Purpose Addons

These addons are used in nearly all my projects, I often install and configure them right after I’ve created a new project.

ember-a11y-testing

ember-a11y-testing is a quality assurance addon that integrates into the existing Ember test framework, adding tests that check for accessibility problems. The addon leverages the wonderful axe-core library to test for form labels, high contrast colors, ARIA attributes and much more.

ember-auto-import

Ember apps can import standard NPM libraries, but it wasn’t straightforward, until now.

ember-auto-import is included with the new Ember Octane edition. It is a build time addon with optional runtime lazy-loading that enables developers to use import statements from standard NPM packages without having to wrap it in an Ember addon or manually wire it into the build by adding it to ember-cli-build.js

Now we can just:

$ ember install ember-auto-import
$ yarn add -D lodash-es

and in your code:

import { capitalize } from 'lodash-es';
let nameUpper = capitalize('edward faulkner');
// Edward Faulkner

ember-cli-update

ember-cli-update is a build time addon that adds the update sub-command to the Ember CLI. This command incrementally updates your app or addon to the latest Ember CLI version. It does this by fetching the latest version and comparing it to your project’s Ember CLI version. It then applies a diff of the changes from the latest version to your project. It will only modify the files if there are changes between your project’s version and the latest version, and it will only change the section necessary, not the entire file.

This tool gets regular use over the lifecycle of all the Ember apps I maintain.

ember-cli-code-coverage

ember-cli-code-coverage is a quality assurance addon that runs at test time. The addon introspects the code running during tests and analyses which code branches were run and how often. It generates a report showing your source code and which branches are covered by tests. This allows you to write test code that covers all code branches. The report looks something like this:

3x  export function asset(param) {
8x    const rootURL = config.rootURL ? config.rootURL : '';
8x    return `${rootURL}${param}`;
    }

Coveralls is a code-coverage-report-as-a-service provider that tracks your projects code coverage over time. The tool auto-uploads your coverage report after tests are run.

ember-cli-dependency-lint

ember-cli-dependency-lint is a build time addon that will lint your app’s addon dependencies, making sure your app or addon has only one version of any dependency. If your app has multiple versions, which is actually packaged in the final build? This situation can lead to anything from hard exceptions to subtle behavioral bugs.

my-app
├─┬ ember-modal-dialog
│ └── ember-wormhole@0.3.6
└─┬ ember-power-select
  └─┬ ember-basic-dropdown
    └── ember-wormhole@0.5.1

This addon will throw a build error until the developer explicitly resolves the dependency either by pinning the dependency with a resolution, or updating the addon with the outdated dependency.

ember-cli-deprecation-workflow

ember-cli-deprecation-workflow is a runtime addon that comes in handy whenever you are updating Ember.js or Ember Data, you’ll eventually get deprecation warnings that need to resolved. In the meantime the console is filled with deprecation warning noise.

The addon listens for deprecation warnings in the console, and adds each to a list. It allows you to add each deprecation warning to a config file where you can work through resolving the deprecations one-by-one.

ember-cli-document-title

ember-cli-document-title is a runtime addon that lets you update the document <title> on a per-route basis. It lets you define the title directly on a title field on any route:

// app/routes/post.js
export default Ember.Route.extend({
  title: "A fresh new post 🥖",
});

This addon should probably be part of Ember core.

ember-cli-dotenv

ember-cli-dotenv is an addon that allows your app to consume environment variables at build time, as defined in a .env file:

API_HOST=https://api.webapp.dev
API_TOKEN=CHANGEME
API_SECRET=CHANGEME

This addon reads these variables and exposes them through the built-in config/environment.js that you can then import in your app wherever you need them. You might want to make your API endpoint URL, port, or any secrets configured as environment variables.

ember-cli-template-lint

ember-cli-template-lint adds lint tooling for handlebars templates to the ember-cli. The lint rules are also automatically added to the test runner so the rules are tested against during ember test runs. This addon was recently integrated into Ember core as a default addon.

ember-test-selectors

ember-test-selectors helps you write tests that look more semantic, and with less churn between refactors. This is because the pattern leads you do bind your tests to a test selector that shouldn’t change during a refactor (instead of directly to an HTML tag).

If your component produces HTML markup like this:

<!-- before ember-test-selectors -->
<h1>{{post.title}}</h1>

<!-- after ember-test-selectors -->
<h1 data-test-heading>{{post.title}}</h1>

You’d change your testing patters to something like this:

// without ember-test-selectors
assert.dom('h1').hasText('Example text'); /* you've explicitly tied the test harness
                                           * to an HTML tag <h1>.
                                           */

// with ember-test-selectors
assert.dom('[data-test-heading]').hasText('Example text') /* Now you're free to change
                                                           * the HTML tag under test to
                                                           * anything, and the test
                                                           * assertion is less context
                                                           * dependent.
                                                           */
                                                           ```

The test hooks, bindings, and data are removed from production builds leaving
your live code running quickly, and clean of any test artifacts.

### ember-truth-helpers

ember-truth-helpers is a runtime addon adding a set of useful handlebars
template helpers for additional truth logic, useful in `if` statements:
`eq not-eq not and or xor gt gte lt lte is-array is-empty is-equal`.

### qunit-dom

One of my favorites, qunit-dom is a quality assurance addon that makes for more
elegant test assertions against DOM elements. Given the following rendered Ember
component:

```html
<div class="ember-view">
  <p class="copy">This is great</p>
</div>

and some example test code, before and after:

// before qunit-dom
assert.equal(
  this.element.querySelector(".copy").textContent.trim(),
  "This is great",
);

// after qunit-dom
assert.dom(".copy").hasText("This is great");

Combining qunit-dom with ember-test-selectors makes for even more powerful test assertions that are independent of the DOM, and all test artifacts are stripped from production builds.

<div class="ember-view">
  <p class="copy" data-test-copy>This is great</p>
</div>

// combining qunit-dom and ember-test-selectors
assert.dom('[data-test-copy]').hasText('This is great')

eslint-plugin-ember

eslint-plugin-ember adds Ember-specific ESLint rules to your ember app, and comes with a great set of recommended defaults that are updated as the community coalesces around best practices. This addon will keep your app code looking clean and tidy, and nudges the code along a path of best practices.

This addon was integrated into the core as a default addon in Ember 2.18.

eslint-plugin-prettier

Prettier is an opinionated code formatter, and eslint-plugin-prettier runs the format rules as an ESLint rule; it reports differences as errors or warnings as ESLint issues. This allows you to catch formatting discrepancies in your editor as well as in CI builds. Prettier rules can be applied automatically in your editor, or by ESLint using eslint --fix.

prettier

Prettier is not an Ember addon, but a package I use in all my Ember projects. Prettier is a code formatter that can automatically format the code in your editor (vim, VS Code, etc all have plugins) as well as detecting formatting issues test time. Super useful for teams trying to maintain a consistent looking codebase.

Specific Usecase Addons

The following addons may not be generally applicable to all applications. I may not use them in all my projects but, when I need the capabilities they offer, then I will reach for them. For example, if I need to authenticate users with an OAuth provider like Facebook or Twitter, I use ember-simple-auth.

ember-cli-addon-docs

ember-cli-addon-docs is a build time addon that creates an interactive sandbox for versioned addon documentation. This tools is wonderful for addon (and app developers!) who need to maintain versioned documentation. This addon lets the developers write their code and not have to think about how the documentation is presented, generated, versioned. The addon has its own code syntax highlighting, interactive live demos, and more.

ember-cli-bundle-analyzer

ember-cli-bundle-analyzer is a build-time tool that creates a visualization chart allowing you to view the size and contents of an app’s bundled output, with their relative and overall sizes. This tool is essential for determining which packages have the largest impact on the deployment size of your application payload.

ember-cli-deploy

ember-cli-deploy is a build time addon that implements a deployment pipeline to upload and activate your Ember app on a variety of hosting providers. It uses a plugin architecture allowing you to find a plugin to deploy to AWS, GCP, and many many more.

ember-cli-mirage

ember-cli-mirage is an addon for mocking backend API responses on the client. It runs in both development mode and testing mode to help you write, test, and prototype your app without forcing you to write the backend parts first. The killer feature is how it unlocks your ability to do FDD (frontend driven development)— you can build frontend features that don’t depend on a backend by mocking the backend in mirage. You can continue modifying mirage requests alongside your frontend feature. Then when the frontend is feature complete, you have established a set of clear patterns you can then implement on the backend of your choice. Mirage is also used during testing, intercepting and mocking API requests instantly so the tests can run more quickly, without making any external network requests.

ember-cli-page-object

ember-cli-page-object is a test-time addon making it easy to follow the page-object pattern by Martin Fowler. Page objects allow you to define the shape of a DOM page in an Ember acceptance test or integration test. They tell your code what to interact with, but do not make assertions about those objects. Your tests then reuse this object. This reduces duplication of DOM selectors in your tests, making your test code less fragile and much easier to refactor.

ember-cli-release

ember-cli-release provides a CLI for building your package, auto-incrementing version numbers, and publishing the packages to a package repository. This is more useful for Ember addons that typically conform to semver conventions, but can be used for engines and applications equally.

ember-cli-typescript

ember-cli-typescript is a build time addon enabling you to write TypeScript in your Ember apps. It hooks into the build pipeline and transpiles TypeScript to JavaScript, and also supplies many of the type definitions for the Ember source code.

ember-cli-fastboot

ember-cli-fastboot is an infrastructure addon. It enables server side rendering of your Ember application. FastBoot runs your application in Node.js so when a user visits your site, they are delivered a fully rendered static HTML page, and only after the content has loaded do they start downloading JavaScript. Once finished, your Ember app takes over, delivering a typical SPA experience. The best of both worlds, but not without complexity.

ember-cli-fastboot-testing

ember-cli-fastboot-testing is a quality assurance addon that allows you to write tests against your Ember app running in the Node.js server context. Since FastBoot runs Ember not in the browser but in a Node.js process, the standard Ember testing tools don’t apply. This addon lets you write acceptance tests that can excersise code running on the FastBoot server.

ember-concurrency

ember-concurrency is a runtime addon that makes it easier to write asynchronus code that support cancelation, restarting, expose their internal state, and much more. This addon makes the difficulty of managing state transitions much easier.

ember-css-modules

ember-css-modules is a built time addon enabling you to write component-oriented CSS. Your styles become private to a component, route, or controller, with explicity features to allow style sharing and composition. It works by making each CSS file its own isolated namespace by transforming class names to ensure they’re unique. Your .css files are now peers of your component .hbs and .js files.

ember-intl

ember-intl is a runtime addon for building internationalized Ember apps, has helpers for tranforming strings into multiple locales, formatting for messages, date/time formats, number, and relative time manipulation. Provides translations using the ICU Message Syntax standard with pluralization support.

ember-intl-analyzer

ember-intl-analyzer is a quality assurance addon providing tools to help you analyze an internationalized app and will locate unused translations.

ember-fetch

ember-fetch is a runtime addon providing an alternative to AJAX requests. It allows the app to make network requests. ember-fetch is useful for applications needing to support network requests in both a browser and FastBoot context, and do not need to depend on jQuery.

ember-power-select

ember-power-select is a runtime addon providing a powerful, and extensible <select> dropdown component that is highly customizable.

ember-simple-auth

ember-simple-auth is a runtime addon for implementing authentication and authorization. It maintains an authenticated client side session, authorizes network requests, and provides helpers in authentication flows like OAuth.

ember-svg-jar

ember-svg-jar is both a build and runtime addon that improves the developer ergonomics around working with SVG assets. It adds a build-time pipeline that discovers SVG files in the project repo and packages them as Ember templates. These are then inlined into your app with an Ember template helper <SvgJar "[ASSET_NAME]"> that embeds the SVG directly into the markup of your page, without an additional network request. It also has a beautiful visual directory route displaying all the SVG assets in your app.

Conclusion

The above addons should cover most of your Ember needs. I have only listed addons with good documentation, test coverage, and that continue to be maintained. If you need anything more or want to take a general look at some of the addons that have are available I recommend starting with emberobserver.com.

Gibsons “San Francisco’s Slow-Motion Suicide” is worth reading, his take is that San Francisco has become too successful… inevitable to decline into some expensive backwater due to “the rent is too damn high”.

He paints a compelling vision of the ills of appalling levels of income inequality, the monoculture of white-male-American tech-workers, and the metastasis of the city into the “hinterlands of Antioch and Vallejo”: the resulting 3 hour-long commutes: yet all these are (in his argument) the results of a conservative set of “not in my backyard” zoning laws that haven’t kept up with the living force of the city.

Gibsons most powerful argument was wonderful:

The legendary urbanist Jane Jacobs once remarked that new ideas come from old buildings, the types of places you can alter without permission because no one cares about them. This is one reason why so many garage startups and garage bands and artists spilling paint in discarded warehouse lofts have left their mark on the world. The true creative class can’t afford to rent expensive new studios.¹

Gibson alludes to the conservative zoning laws and wealthy property owners exacerbating the plight of San Francisco and similar cities: Owners want property values to rise above all other concerns. The problem in San Francisco isn’t a lack of development, but the opposite: The lack of LIFE itself. San Francisco has privileged the physical static object of the real-estate-object over the use of space in time. The people owning property and businesses privilege the real estate value over use value, and the result is a static dead space where object-value dominates and eliminates the use of space for daily life.

Recently a friend of mine packed his motorcycle up with his distillation laboratory, plastic molding equipment, machine shop tools, his immense library, his formidable liquor collection, and moved down to Philadelphia, where he can have a far larger living space, workspaces are abundantly available, and where the barrier to start new businesses and develop existing vacant lots for new uses are minimal or non-existent.

The drive to urbanization is an imperative of capitalism, this drive results in a freezing of life—a static world devoid of spontaneity, the result of land development and bureaucratic zoning laws optimized for rent seeking and increased land valuations, devoid of nature and where its residents have no ability to change their surroundings to suit daily life. All desire for change is eliminated in order to serve the profit generating imperative of capital, all autonomy of local residential use is therefore foreclosed, instead privileging real estate development as the only legitimate use.

The result of this is a predominance of land organized around the valuable placement of real estate objects over the living use and alteration of land by the very people who live in these spaces, people who wish to change their surroundings over time as use-patterns and ways of living change with the seasons and with time. The philosopher Hegel explored this concept saying:

“Space is the connection of the quiescent asunderness and side-by-sideness of things [objects]; Time is the connection of their vanishing or alteration… In the spatial world the question is not succession but of coexistence… As a restless Becoming [Time] is not an element of the synthetic whole.”

This is to say: the privileging of objects in space fetishizes the object and therefore sacrifices changes in time. This creates an alienated and static space of purely fetishized structures, devoid of life.

This was an excerpt from “The Society of The Spectacle”² a forthcoming translation of Guy Debord, translated by 0xADADA.

All modern browsers increasingly support a standard called the WebExtensions API. This API provides extensions the additional functionality to add or change the core features of the browser. Extensions developed to follow the WebExtensions API will in most cases run in Firefox, Chrome, Opera, Brave, and Edge with just a few minor differences.

An extension will typically be a composition of any or all of the architectural parts of the extensions API:

• background scripts Are essentially JavaScript running in a hidden tab, loaded when the browser starts and run continually. Background scripts are where extension authors write code that handle global browser events and respond to actions. They maintain long-term state, or can perform long-term operations. This code runs independently of any any particular web page or browser window.
• content scripts Are JavaScript and CSS loaded into any web page whose URL matches a specified pattern. This code can be written to modify web pages (e.g. AdBlockers), or add features to web pages.
• browser actions A button and icon that the extension adds to the browser’s toolbar that can trigger events. These actions may have a popup.
• popup A popup is really just another hidden tab which is shown when the extensios toolbar icon is clicked. The content of a popup is specified using HTML, CSS, and JavaScript. popup.html (in blue, below) is a basic html page that is loaded into the popup window, but can also be opened in another browser tab just like any other html page (if you know the extension URL).
• page actions Similar to browser actions, but these can add menu items to the on-page context menu, and can also open a popup window.
• options pages An extension can also specify additional pages that are available as popup windows, often offering extension preference panels.

Creating a simple extension

Every extension must, by definition define a manifest.json file, which is the only file that every extension using WebExtension APIs must contain. The manifest.json specifies basic metadata about an extension, and also resolves how the code in the extension is loaded, including content scripts, background scripts, and browser actions.

In order to work in all modern browsers, a few fields must be defined in manifest.json:

• manifest_version defines the WebExtensions API level (2 is the current version)
• name the extension name
• version version of the addon
• author name of the person or organization (required by Edge) who wrote the extension

A few other key fields are optional but highly recommended:

• description text displayed in the Extension UI and in the extensions app stores
• icons will be shown in the browser toolbar, in the Extension manager UI, and the various extensions app stores
  • a 32px icon will be shown in the toolbar (resized)
  • a 48px icon will be shown in the Firefox Add-ons Manager and Chrome Extensions Management
  • a 96px icon will be shown in the Firefox Add-ons Manager (on high DPI devices with a Retina display)
  • a 128px icon will be shown in the Chrome Web Store

To deliver the best visual experience to users with high-resolution displays, the browser will attempt to find the best matching resolution icon. It is best practice to provide a set of icons in multiple resolutions.

This manifest.json defines a minimal browser extension that provides an icon but doesn’t do anything, but hey, its something:

{
  "manifest_version": 2,

  "version": "1.0",

  "name": "My extension",
  "author": "me",

  "description": "My first extension",

  "icons": {
    "16": "icon-16.png",
    "32": "icon-32.png",
    "48": "icon-48.png",
    "96": "icon-96.png"
  }
}

Mozilla maintains a WebExtensions browser compatibility table which is helpful for getting the manifest.json working across all major browsers.

Extension Runtime Contexts

Access to the complete WebExtensions API is not universally accessible from JavaScript. There are different levels of access depending on the context in which the code is running. There are essentially two contexts: The background context which has access to the complete WebExtensions API, but cannot access any web page DOM; and the content script context, which has access to the any web page DOM API, but doesn’t have complete access to the WebExtensions API. The two contexts work together with a message passing API that allows them to accomplish tasks together.

Background scripts run in the context of a special hidden tab called a background page. This gives them a window global containing parts of the standard DOM API. Background scripts can access all of the WebExtension APIs (as long as the manifest.json has opted the extension into the necessary permissions). Background scripts cannot access the DOM of any web page.

Background scripts can make XHR requests to any hosts (as long as the manifest.json have granted host permissions). Background scripts do not get direct access to web pages, however, they can load content scripts into web pages and then can communicate with these content scripts using the browser.runtime.sendMessage API. The content scripts can then access the web page DOM on behalf of the background script.

Content scripts are extension-provided scripts which run in the context of a web page. These scripts can see and manipulate the page DOM, just like normal scripts loaded by the page. Unlike normal page scripts, they can also make cross-domain XHR requests (when granted permission). Content scripts have access to a more restricted subset of the overall WebExtensions API. For example, the browser.cookies API is not available to content scripts. Content scripts can exchange messages with their background scripts, and in this way indirectly access all the WebExtension APIs.

Web extension code communicates across contexts using either the standard DOM API window.postMessage(...): for communicate between content scripts and other scripts on the host page, or the content scripts and other page scripts can communicate with the background script using the browser.runtime.sendMessage(...) API. The background script can additionally send messages to the content script using browser.tabs.sendMessage(...).

Deploying an extension

Extensions are compressed into a zip file before being uploaded to the extension app store. Mozilla maintains a build tool for creating a zipped package that’ll work on both the Chrome Web Store and the Firefox Addons store. This tool can also perform project linting, as well as testing and a development runner that hot-reloads extension code for Firefox.

• web-ext
• Documentation

Resources

• Google WebExtensions API
• Mozilla WebExtensions API
• Brave
• Microsoft

“Opus” is a brilliant postmodernist introspection of the challenges of writing, creativity, and the cliches of pop culture; delivered in a manga format.

Author Satoshi Kon worked on this manga between 1995-1996 as it was serialized in “Comic Guy” manga magazine until the magazine closed abruptly in 1996. The final 3 chapters were never finished. Between 1996 and his death in 2010, he sketched out an abridged final chapter that was included in this book.

The brilliance of this book is how the final chapter serves as a bookend to his life, how our individual lives are stories left unfinished, and how our loved ones go on to continue those stories. A true gem.

”How would you make this work?”

add(2, 5); // multiple arguments style
> 7
add(2)(5); // currying, a chained invocation style
> 7
add(1,2)(3,4) // both
> 10

I thought this was a very interesting question, so took some time to implement a multi-argument / curried function.

Heres what I got:

A solution to the multiple argument style is straightforward, theres just the bit to change arguments into an Array, and to handle the case when nothing is passed in.

let sum = function () {
  let args = arguments.length ? Array.from(arguments) : [0]; // convert arguments to array for .reduce
  return args.reduce((acc, i) => (acc += i));
};

Now, to get chained invocation style to work, I used bind to generate a new function that would be returned to the caller, allowing for chained invocation. The tricky bit is to set the valueOf function to return the sum, so when checked for a value, the function returns a number.

Thus:

let sum = function () {
  let args = arguments.length ? Array.from(arguments) : [0]; // see (a)
  let sum = args.reduce((acc, i) => (acc += i));
  let f = sum.bind(null, sum); // see (b)
  f.valueOf = () => sum; // see (c)
  return f;
};
/* (a)
 * [0, ...arguments] will convert arguments to an Array
 * to allow the `reduce`. It will also create an initial item
 * `0` to handle the case no arguments are passed in. Thus
 * making `sumChainable()` possible.
 *
 * (b)
 * Generate a nested arrow function that will be returned, and pass
 * the sum to it. This allows the return value of the function
 * to be invoked in a chain, each changed invocation passing
 * the sum of its caller. Thus `sumChainable()()` is possible.
 *
 * (c)
 * Setting the `valueOf()` function on the returned function to
 * return the sum allows the comparison operator `==` to check
 * the value of the function against a number. Thus making operations
 * like `sumChainable() == 0` and `sumChainable(1, 1) + 2 == 4` possible.
 */

We can clean up this code by using the ... rest parameters.

function sum(...args) {
  const value = args.reduce((acc, a) => acc + a, 0);
  const f = sum.bind(null, value);
  f.valueOf = () => value;
  return f;
}

added a function to run test conditions, and defined some tests:

function test(conditions) {
  conditions.forEach(([msg, cFn], i) => {
    const result = cFn();
    console.log(`${i} ${result ? "✅" : "🚫"} ${msg}`);
    console.assert(cFn(), msg);
  });
  console.log(`${conditions.length} tests completed`);
}

let conditions = [
  ["sum() returns 0", () => sum() == 0],
  ["sum()() returns 0", () => sum()() == 0],
  ['typeof sum() returns "function"', () => typeof sum() == "function"],
  [
    'typeof sum().valueOf() returns "number"',
    () => typeof sum().valueOf() == "number",
  ],
  ["sum(0) returns 0", () => sum(0) == 0],
  ["sum(0)(0) returns 0", () => sum(0)(0) == 0],
  ["sum()(1) returns 1", () => sum()(1) == 1],
  ["sum(1,2) returns 3", () => sum(1, 2) == 3],
  ["sum(1,2,3) returns 6", () => sum(1, 2, 3) == 6],
  ["sum(1,2,3)() returns 6", () => sum(1, 2, 3)() == 6],
  ["sum(1,2,3)(1) returns 7", () => sum(1, 2, 3)(1) == 7],
  ["sum(1,2,3)(1) returns 7", () => sum(1, 2, 3)(1) == 7],
  ["sum(1,2,3)(1,2) returns 9", () => sum(1, 2, 3)(1, 2) == 9],
];

test(conditions); // run the tests

viola!

Future improvements would be to get strict equality === operator to recognize the result as type number rather than type function, but Javascript doesn’t have a way to override operations. We can also change the API slightly by invoking .valueOf() at the end of the invocation chain to yield a number primative value.

// test for strict equality using .valueOf()
console.log(
  `sum(1,2,3)(4,5)(6).valueOf()`,
  sum(1, 2, 3)(4, 5)(6).valueOf() === 21 ? "passed" : "failed",
);

future would would be to get working:

console.log(
  `typeof sumChainable(1) === 'number'`,
  typeof sumChainable(1) === `number` ? "passed" : "failed",
);

Alarm goes off, time to wake up. Snooze alarm, and then repeat: check email, check Facebook, check Twitter, check Snapchat, rinse; repeat again; no new posts? Check again. No second spared to compose a thought; dreams fade away. Day continues just the same way — rise, rinse, repeat, repeat. Can’t spare time to sit and relax, can’t let your mind wander not five minutes, no unstructured thought or daydream; just repeat. Every moment captures value — must increase engagement! must repeat! —no time for grander narratives, to solve for bigger problems. Atomize attention into smaller slice of microwork, then catch up, read, listen, swipe right, repeat, like, pull-to-refresh, to scroll to infinity, and then again. No matter if you’re pulling espresso shots, serving tables, or playing with computers all day: they all end the same: Netflix & chill, the guardian of sleep, TV, our collective lullaby. Wake up with screens, go to bed with screens. We no longer shit alone. The death of idleness.

This essay explores the ways Facebook transforms our attention into a product, and how that transformation changes us. It then proposes a social media strike as a concrete strategy to reclaim our attention, and finally lists many reasons why we should all quit Facebook.

The Commodification of Microwork

Social media companies like Facebook, Google, Twitter and Snapchat have created a marketplace for our attention, where we become the product sold to advertisers who subtly change our behavior to buy products and services for their own benefit. This type of persuasion is most effective when these platforms command our attention with a heightened level of distraction. We create the content that is then used to command the attention of our peers, which is then used to sell advertising space while the surrounding content is measured for engagement, and the results are analyzed to optimize the next round of ad placement. This process is the commodification of microwork— the seemingly small tasks that we are compelled to do by force of habit, tasks that aren’t in our own interests but in the interests of the platforms and advertisers using our attention and time to pad their bottom lines. Also known as “heteromation” (Ekbia, Nardi, 2014), microwork includes reading timelines, posting photos & updates, liking, retweeting, and generally dedicating our time, attention, and emotions to these platforms—each tiny action its own form of labor, given over freely to the advertisers.

The new media of surveillance capitalism solicit social behaviors, monitor those behaviors, map social interactions, and resell what they learn to advertisers and others.

Platforms like Facebook are attempting to create systems for the generation of distraction, Crary describes it as follows:

conditions that individuate, immobilize, and separate subjects, even within a world in which mobility and circulation are ubiquitous. In this way attention becomes key to the operation of noncoercive forms of power.

Over time, these systems continually push our attention and distraction to new limits and thresholds. Youtube & Netflix discovered that auto-playing the next related video dramatically increases views (Bridle, 2017); Twitter uses the “pull-to-refresh” UI gesture that leverages variable rewards to trigger addictive behavior as a way to increase user engagement with the timeline; Facebook uses the “red dot” notifications that keep people checking their phone for the next new thing. These gadgets are designed to create a “supernormal stimulus,” that is, a stimulus that produces a stronger than natural response. We can even internalize the supernormal stimulus—an example is the “phantom vibrate” we sometimes feel in our pocket when no vibration occurs. In the case of social media, the supernormal stimulus is used to exploit our response to novelty in order to elicit a behavior that works in the interests of the social media provider.

This eventually leads to a crisis of attentiveness, where the system must maintain an increased level of distraction. In order to continually distract us, the visual landscape must constantly change, requiring us to reorient our attention until the shift from one thing to another becomes the natural state (e.g. the Instagram “Explore” tab). These systems are training our brain to glance and skim, reading only a few sentences, watching increasingly shortened clips, and continually shifting focus to the next snippet in a continual route to novelty instead of focus.

Over time, we could begin to value our attention while the platforms struggle to get more and more of what they previously got for free. This may not be sustainable, as newer products need to continually revolutionize the means of distraction else we realize how distracted we truely are. It is only through a return to focus that we may ever achieve self actualization.

This process of commodification has turned us all into tastemakers, reviewers, likers, retweeters and brand ambassadors. The platform takes our real authentic friendships and first commodifies them, reifies them, and then sells them back to us as an “image of friendship”, but one that is bankrupt of any genuine social value. Over time, these platforms transform us all into unpaid advertising agencies. We promote goods, services, lifestyles and desires to our friends, weaponizing images to generate feelings of jealousy and FOMO amongst our peers during those idle moments when they feel most bored. These idle moments are when we are most vulnerable, and thus we’re psychologically primed to accept the supernormal stimulus. The platform capitalizes on this vulnerability, and over time begins to redefine what we once considered “real” friendship into a relationship with the platform itself, mediated by the features and “images of friendship” within it through Liked and Retweeted posts, Snapchat streaks, Follow requests, posted text, images or even how we reduce our emotions into a series of emoji (Smith, 2016).

In this way, the platform is able to monetize our friendships, tastes, opinions, and even our emotions. Our internal thoughts and experiences become commodifiable assets, measured as engagements to be analyzed, A-B tested, optimized and charted, then touted by executives in PowerPoint presentations at board meetings. Our experiences are mined, packaged, and sold, and we are not paid a dime.

No rest, no meditation, no reflection, no conversation—the senses are continually overloaded with stimuli. Man Doesn’t learn to question his world anymore; The screen offers him answers already made.

Instead of blindly activating our social media habit for that quick dopamine fix, consider who actually benefits. Are you performing a microwork task for Facebook or Snapchat? If so, then why aren’t you being paid?

What They Say vs What They Do

Newspapers, cable news, and social media platforms are trying to turn us all into media addicts. These businesses rely on persuading us to consume more and more (keyword: “increase engagement”) in order to curate an “interest in current affairs” (e.g. New York Times) or to “build meaningful and connected relationships with our community” (e.g. Facebook.)

The reality of the media, however, is driven by a simple business rule: Sell what sells best, the supernormal stimuli: whether it’s clickbait, memes, curated lifestyles, celebrity gossip, salacious headlines, and freak events. In the drive to maximize profit, these organizations are required to maximize content engagement, and the kind of content engages users the most is content that enrages, shocks, content that’s absurd, grotesque, weird, or funny, content that exposes the horrible, or content with threatening policies. The media tends to focus on the rare incidents that have no actual influence on our daily lives, and the goal of these platforms is to keep us addicted. This attentiveness to negativity is then reinforced to the point of addiction, and addiction to negativity turns to depression and teaches us learned helplessness. Quoting media researcher Dobelli “When we tune into the news, we are constantly confronted with unresolved problems and the narrative does not inspire much hope that they will ever be solved.” (Dobelli, 2020)

When it comes to engagement, Facebook’s 2017 Annual Report makes it clear:

The increase in the ads delivered was driven by an increase in users and their engagement and an increase in the number and frequency of ads displayed on News Feed, partially offset by increasing user engagement with video content and other product changes.

We think we’re immune to the persuasiveness of advertising, but remember, advertising is a $600 billion dollar industry that is the financial basis of the tech sector. The massive profits these companies have built have been done so on the backs of advertising campaigns.

Facebook’s (and Google’s, etc.) real agenda is to display and sell ads—not to help people “build meaningful and connected relationships.” At no point was this more clear than during Mark Zuckerberg’s testimony before Congress. When Senator Hatch asked him, “How do you sustain a business model in which people don’t pay for your service?” his response was as straightforward as it could’ve been: “Senator, we run ads.” (NBC News, 2018).

“Building Relationships” is the rationale used in order to serve the advertisements, but the advertisements themselves are the real content of these platforms. The business arrangement of the platform is a simple one: ads are the content, the advertisers are the clients, the user doing microwork tasks are the workers, and our future behaviors are the product.

As a result, we end up building relationships to the platform—not to each other.

Through the investment of our attention, we allow the commodities and ideologies of the platform to bombard our unconscious and subtly shape our behavior. The process of “exchanging time for image, provides the counterflow to the moving image and as advertising revenues would indicate, is itself productive of value.” (Beller 2006, p.79)

Facebook’s nearly one billion users have become the largest unpaid workforce in history.

The intended effect the media wishes to create is media-driven neuroticism—a love of change for its own sake, or neomania, a love of new things. Ultimately, neomania together with the media has a negative effect on our moods, which isn’t surprising considering most of what makes the daily news is negative. This leaves us feeling powerless, like the world is falling apart. So we turn to social media to witness the (seemingly) beautiful lives of celebrities, and the romantic and exciting lives of our friends and family. But this ends up driving the compulsion to compare our own lives with those of the people we see on social media, creating feelings of inadequacy, loneliness, and jealousy. These feelings are misguided however, since the lives we see in the media and social platforms are a tightly edited and curated spectacle of people presenting an image of how they want to appear, not of how they actually are.

Practical Defense Against Unpaid Microwork

Today it’s popular to hear about the media detox or a social media diet. The concept is often framed as a metaphor for eating and weight loss. We can extend this metaphor to consider the microwork tasks we find ourselves doing on a daily basis. Are the habits and media and we consume daily like the healthy foods that we can use to build a healthy mind and body? Or will they ultimately poison us? To live a healthy life we should moderate our media consumption the same way we do with the things we eat.

The toxicity of the media can be understood as a signal-to-noise problem. The more media we consume, the more noise we get (rather than the valuable part, called the signal.) If we consume the news on a yearly basis (in the form of books), we can assume about half of what we consume is signal, and the rest noise (randomness). If we increase our frequency to a daily intake, the composition would increase to about 95% noise, and only 5% signal. If we increase our intake to an hourly frequency (as those who follow the stock market or heavy social media users do) the composition increases to 99.5% noise and 0.5% signal. This is roughly 200% more noise than signal, more toxic noise to both misdirect and distract, and has negative effects on our mood.

Distraction could be described as a phenomenon in which you connect, inadvertently or absentmindedly, to more things than you intended. The noise drowns out the signal.

A practical approach for a healthy relationship with the media we consume is to create distance between ourselves and the media that surrounds us on a daily basis. Distance creates an opportunity to gain new perspectives on the media (and life in general). This distance reduces the influence of these platforms, and creates a space for our ideas and imagination to flourish.

Idleness and imagination are essential to our wellbeing, they’re the wellspring of mental clarity, and the backstop for memory. Take a few minutes each day to cultivate idleness and some mental space. Step away from the hourly, daily, weekly media cycle, and read books instead. Books benefit from the perspective of time, which results in more settled and established facts with increased depth of analysis. If our goal is to have a wider perspective, and a balanced understanding of the major forces at work in the world, our best bet is to read good books.

This essay gives us the task of avoiding contact with the mundane, cheap, and ephemeral media that surrounds us on a daily basis; to avoid having our attention distracted, redirected, exploited, and abused against our own interests. We should ignore those things that bring no happiness or value to our lives. Instead, we should focus our attention on our imagination, quality thought, friends and family, idleness; on things that matter. We should explore effective ways to reclaim clarity of thought, and autonomy of attention.

The task is to change the very way we attend the world around us. Henry David Thoreau described a similar mental recalibration in Life Without Principle:

If we have thus desecrated ourselves, — as who has not? — the remedy will be by wariness and devotion to reconsecrate ourselves, and make once more a fane of the mind. We should treat our minds, that is, ourselves, as innocent and ingenuous children, whose guardians we are, and be careful what objects and what subjects we thrust on their attention. Read not the Times. Read the Eternities.

This essay proposes the survivorship bias as the criteria to determine which media to ignore as mundane, cheap, and ephemeral. Survivorship sets the focus on knowledge that has withstood the test of time; Nicholas Taleb makes the following recommendation in Antifragile:

“[read] as little as feasible from the last twenty years, except history books that are not about the last fifty years. […what most people do] is read timely material that becomes instantly obsolete.

Aaron Swartz had similar sentiments:

Its obsession with the criminal and the deviant makes us less trusting people. Its obsession with the hurry of the day-to-day makes us less reflective thinkers. Its obsession with surfaces makes us shallow.

Survivorship is a simple heuristic with which to judge media, be it books, news, social media, television, academic journal articles or even video games. A clickbait article can go viral and get millions of views the first day it’s published, but according to the survivorship bias, its newness means that it’s likely to be obsolete the next day. Rather than trying to judge for yourself if media is worth your attention, try letting the survivorship bias do the work for you.

The survivorship bias has a few knock-on effects that are worth mentioning. Media tends to be discounted as it ages. Video games are always on sale a few months and especially years after they’ve been released; Books and movies are always cheaper well after their release as well; Newspapers are free the day after they were published; and so on. Beyond saving our attention from instantly obsolete media, we will also be saving money simply by waiting long enough. With time also comes more in-depth analysis from the surrounding community; reviews and deep interpretations of the rich meaning behind film and video games benefit from the additional time it takes for this analysis to blossom. This cultural analysis isn’t available to those following the cult-of-the-new.

Media Consumption Strike

Now that we understand how the territory of our attention has been colonized, we can begin to resist. Our challenge is to put the following strategies in practice until they form new habits that encourage idleness and mental spaciousness. We have the tools to reclaim our time and imaginations, we must take action!

The inadvertent laboring towards the propagation of the spectacle is another example of the ways that the spectacle takes advantage of its subordination of the masses. It is a process to which they have no control other than to renounce image culture completely which in a hyperreal world is almost inconceivable.

Any efforts to reform the practices that produce the addictive symptoms of the attention economy will fail to address the underlying structures of late capitalism that created these markets—namely, market fundamentalism, technologic determinism, deregulation, and privatization (Tanner, 2018).

Given that; we propose a collective media consumption strike, and rather than allowing our attention to be driven by these platforms, we must direct our attention to the things that matter to us.

The rules can be simple:

• No advertising-sponsored TV or Youtube
• No commercial radio, news, or news aggregation
• No commercial social media, delete your Facebook account

This is not a holier-than-thou asceticism, but a refusal to have our attention destroyed and exploited. Don’t be a fundamentalist about the rules. Rules are meant to be broken. If it’s art, the rules are more loose; if it’s corporate media, more strict. Cinema, literature, sci-fi, comic books, indie games (or board games): as long as these types of works are not connected with our professional, interpersonal, or political responsibilities, use them in moderation. In general, make sure media consumption is done with more moderation than usual. The goal is to break your habits.

Many people work with computers for their job. Most aren’t privileged enough to be able to completely disconnect from work. Work-related email, messaging, and work-related browsing are out of scope for the strike by necessity—but do please try to prevent any non-work related web browsing or media consumption. Don’t use work as an excuse to view social media or browse the web. Academic journal articles and the like will be hard to avoid, but ask yourself if they would pass the survivorship bias after a few years.

Enable Do Not Disturb on your phone. This will establish designated mental space for ourselves, friends, and family. Most smartphones have rules that can automatically put the device into Do Not Disturb mode at set times. In Android and iOS, this is available by going to Settings and searching for Do Not Disturb preferences. Do Not Disturb is also available in macOS as well.

Remove anything that provides variable information rewards. These are features in apps and games designed to give a random, pleasant surprise—to create addiction. Slot machines are designed to do this; social media platforms like Facebook, Twitter, and video games like Candy Crush use “loot crates” to achieve the same result. Email can operate similarly. Turn off the red Notification dot on all your apps.

Remove Facebook, Snapchat, Twitter, Slack, Instagram, Youtube and any other commercial social media apps from your devices. This makes it tougher to have instant access to these forms of media since it’s not immediately at your fingertips. Don’t cheat and use the web interface. Disable notifications for any apps that you keep. Turn off all lock screen notifications, this also improves privacy. Remove any of the other chat apps from your smartphone: Snapchat, Facebook Messenger, WhatsApp, Mastodon, Viber. If our friends need to reach us, they can text or call. We should be the masters of our own time, and our network of friends isn’t in control of our time. Our inbox isn’t the world’s way to add items to our TODO list.

Remove any Bookmarks or Recently Viewed sites from your browser to prevent 1-click-away sites from distracting us. Some software like SelfControl will block habitual sites for you. Reddit, BuzzFeed, Upworthy, Medium are great sites to block.

Try to limit checking email to once a day. Set a schedule at a set time, and stick to it. Do the same for messaging. Set an auto-responder if that helps reduce anxiety.

Tell your friends why you are leaving commercial social media, make it clear that your reason to leave is political, and not because you’re ignoring them. Give your friends a direct way of reaching you.

The idea behind this practice is to stop the urge to immediately unlock our phone whenever we have idle time. Our idle time should be owned by us.

Conclusion

The net effect these platforms have on us is to alienate us from our very lives. Instead of focusing on what is valuable and good, we tend to focus on what we lack, or what we’re missing out on. If we divest ourselves from this tendency, ignore the rare incidents and ephemeral content that has no actual relevance to our daily lives, the net effect is beneficial to our mind and spirit. The resulting autonomy of attention can allow us to connect to the people and environments around us, and reclaim our very imaginations.

0xADADA is a software developer working for AI and web related startups in Boston; graduated from Northeastern University with degrees in Computer Science and Cognitive Psychology. 0xADADAs’ Facebook account was created in 2004 and deleted in 2015.

Appendix: Reasons to Quit Facebook

The ideological rationale that keeps platforms like Facebook profiting from the commodification of our attention and the collection of our private data lacks common courtesy at best, and is psychopathic at worst. Here we present our Airing of Grievances, a listicle of the top reasons to quit Facebook.

Facebook outsources data exploitation for political manipulation to companies like Cambridge Analytica. (Cadwalladr, Graham-Harrison, 2018).

Facebook has democratized surveillance, where we have normalized reporting upon the intimate details of the status of our friends and family. (e.g. “X got engaged to Y”, “Tagged my friend A in this photo, at location X”, “B works at company C.”).

Without privacy, people resort to self-censorship, and therefore remove any aspect of political action or critique from themselves, thus becoming normalized to political impotence. (Assange, Appelbaum, Müller-Maguhn, Zimmermann, 2012).

Former founding President Sean Parker, along with former vice-president of user growth Chamath Palihapitiya have both objected to Facebook’s use of dopamine-driven feedback loops to increase addictive behavior. (Hern, 2018).

Former Chief Information Security Officer Alex Stamos resigned from Facebook over an internal disagreement in how much Facebook should publicly share about how nation-states used the platform in the run-up to the 2016 elections. (Perlroth, Frenkel, Shane, 2018).

Co-founder of WhatsApp (acquired by Facebook) Brian Acton has said people should #DeleteFacebook. (Newton, 2018).

Co-founder of WhatsApp Jan Koum, is planning to leave the company after clashing with its parent, Facebook, over the popular messaging service’s strategy and Facebook’s attempts to use its personal data and weaken its encryption. (Dwoskin, 2018).

Facebook correlates data from loyalty program providers (e.g. Walgreens cards from Nielsen-Catalina Solutions) who have access to brick-and-mortar purchase history with individual Facebook accounts. Companies like Johnson & Johnson can buy this data from the broker and use Facebook tools to target individual users for ads promoting products they’ve previously purchased. (Stern, 2018).

Facebook has been fined for breaking privacy laws in the EU for using cookies, and social plugins. (Lomas, 2018).

Facebook’s corporate policy is based on growth and engagement at any cost. (Malik, 2018).

Facebook’s Protect VPN product is collecting users’ mobile data traffic and sending it back to Facebook. (Perez, 2018).

Facebook hired a full time pollster to monitor Mark Zuckerberg’s approval ratings and develop strategies to change public perceptions. (Newton, 2018).

Facebook’s network “is large enough and deep enough to create a global census that can ‘see’ nearly everyone on the planet, even if they don’t have a Facebook account.” says John Robb, a former counter-terrorism operative in US Special Operations Command. He goes on to say, this will “enable real-time tracking on nearly everyone on the planet using smartphone GPS data and ancillary information”. (Ahmed, 2017).

“Facebook’s business is to simulate you and to own and control your simulation, thereby owning and controlling you.” (Balkan, 2017).

Facebook has experimented with removing popular news outlets from the Feed in poor countries including Sri Lanka, Guatemala, Bolivia, Cambodia, Serbia and Slovakia. (Hern, 2017).

Former Facebook platform team Operations Manager Sandy Parakilas claims the company prioritized data collection of its users over protecting them from abuse. (Parakilas, 2017).

Facebook Likes can be leveraged to reliably predict intelligence, personality traits and politics. (Hess, 2017).

Facebook uses messages and contact details of our friends and other Facebook users to build a shadow profile of us that makes it easier for Facebook to more completely map all our social connections. (Hill, 2017).

Facebook still knows what you typed before [and after] you hit delete. (Boykis, 2017).

Facebook will tag your face in any photos uploaded from any users. (Boykis, 2017).

Facebook became 45% of all referral sources of traffic between 2014 and 2017. This has an enormous influence on what people see on the web, making the web more centralized upon Facebook. (Staltz, 2017).

Facebook is filing patents to detect and use emotion to influence users’ behaviors. (CB Insights, 2017).

Facebook uses gamification to incentivize us to keep checking how many Likes our posts have gotten. A narcissism reinforcement machine. (Dillet, 2017).

Facebook uses nostalgia, birthdays, and sentimentality to manipulate us to increase engagement. (Frost, 2017).

Facebook’s Like button (seen on almost every site on the web) isn’t just there to make it easy to post that page on Facebook, but it also provides a hook for Facebook to track your visit to that site, and thus collect your entire web browsing history. (Satyal, 2017).

Facebook revealed the identities of its own content censors to suspected terrorists. (Solon, 2017).

When Facebook has trouble acquiring users in specific markets, it simply buys the companies that dominate those markets (e.g. Instagram, WhatsApp and Oculus). (Srnicek, 2017).

Facebook owns a patent to use our devices’ camera to gauge our emotional state from our facial expression, to better to target us with content and advertising. (Dowling, 2017).

Facebook prevents search indexing, so content posted on Facebook is only discoverable within Facebook. (Gruber, 2017).

Facebook fragments the public debate into filter-bubbles, and users segmented into one bubble will never see the news and information in another bubble, separating society along ideological lines. (Lanchester, 2017).

Facebook can identify when teens feel “insecure”, “worthless” and “need a confidence boost” in order to keep them hooked. (Lewis,. Machkovech, 2017).

The data we give Facebook is used to calculate our ethnic affinity, sexual orientation, political affiliation, social class, travel schedule and much more. (Miller, 2017).

Facebook buys personal data from various data brokers, and correlates it to Facebook profile data to build aggregated profiles that span multiple sources. (Angwin, 2016).

Facebook News team suppressed conservative news items. (Nunez, 2016).

Facebook prevented News Team curators from listing Facebook on their resume in order to make the organization seem like it was unbiased and AI-driven. (Nunez, 2016).

Facebook can use your name and photo to endorse products and services to your social network without your knowledge. (Tucker, 2016).

Facebook is the television of the web, letting us passively scroll through content that we’d probably like, based on our habits and things we’ve already Liked, putting us in comfort bubbles that are more isolating than physical walls. (Derakhshan, 2016).

Facebook creates an illusion of choice, but by shaping the menus we pick from, it hijacks the way we perceive our choices and replaces them with new ones that aren’t in our best interests but serve the interests of Facebook. (Harris, 2016).

Facebook Likes, status updates, and pages we visit are “more reliable” in predicting mental illness. (Reynolds, 2016).

Facebook uses intermittent variable rewards, used in slot machines to maximize addictiveness, by linking an action we take (pull-to-refresh) and a random reward (e.g. new posts!). (Harris, 2016).

Facebook convinces us that we’re missing out on something important, when in reality we’re not. (Harris, 2016).

Facebook abuses our need for social approval, validation, and the need to belong in order to increase engagement. (Harris, 2016).

Facebook abuses our need to reciprocate the social gestures of others. (E.g. the need to Friend someone back who has Friended you). (Harris, 2016).

Facebook abuses our attention with immediate interruptions because studies have shown it increases engagement. (Harris, 2016).

Facebook abuses our intentions by hijacking our tasks with the needs of the platform. For example, when you want to look up a Facebook event happening tonight, the app doesn’t allow us to access the event without first redirecting us to the News feed. (Harris, 2016).

Facebook makes it easy for us to handover self-incriminating data that can be used against us by law-enforcement. (Clark, 2016).

Facebook blocked the account of activist Shaun King after he posted a racist email that was sent to him. (Stallman, 2016).

Facebook enforces a real name policy, allowing corporations and nation-states to be able to connect users’ accounts with their real identity. This is dangerous for marginalized people, and makes them vulnerable to blackmail lest their real identities be exposed. The real name policy forces people to have a single identity when in reality people have flexible identities that change depending on social context. Using one’s real name inhibits one from experimenting with alternative identities, limiting personal growth to normative concepts of identity. (Stallman, 2016).

Facebook has censored posts for Israel, Russia, China, Turkey, the UK, and routinely suppresses content for political reasons using algorithmic promotion and depromotion. (Stallman, 2016).

On the web, hyperlinks are freely swapped to enable the cross-pollination of information and a diversity of decentralized ideas. On Facebook, each post exists unto itself, often accessible only within Facebook amongst ones “Friends of Friends”. “instead of seeing [hyperlinks] as a way to make that text richer. You’re encouraged to post one single hyperlink and expose it to a quasi-democratic process of liking and plussing and hearting: Adding several links to a piece of text is not allowed. Hyperlinks are objectivized, isolated, stripped of their powers”. (Derakhshan, 2015).

Facebook hurts the power of the website: “the Stream means you don’t need to open so many websites any more. You don’t need numerous tabs. You don’t even need a web browser. You open Twitter or Facebook on your smartphone and dive deep in. The mountain has come to you. Algorithms have picked everything for you. According to what you or your friends have read or seen before, they predict what you might like to see.” (Derakhshan, 2015).

Facebook (even more so, Instagram) is the cul de sac of the internet. It’s where content can no longer be enriched with annotations external to itself. It’s where conversations wither and content goes to stare inwards at itself.

Facebook analyzes the contents of messages sent between users on the platform to better target advertisements. (Virani, 2015).

Facebook uses friends to gather additional information about us. Tagging friends in photos, answering questions about a friend’s marital status are ways we’re tricked into snitching on our friends. (Virani, 2015).

Facebook encourages us to present normative images of our lives, which results in alienation from the actual way we feel. (Krause, 2015).

Increased use of Facebook is linked to depression. (Wald, 2015).

Facebook’s internet.org project was touted to provide internet connected devices and networks in India, but created a Facebook-only view of the internet. (Burrington, 2015).

Facebook abuses our innate tendencies to track our progress and assess our self-worth by comparing ourselves to other people. (Musser, 2015).

Facebook photo data is used by Nashville company Facedeals to identify shoppers in stores with the stores’ own security cameras and facial recognition software. These profiles are then used to increase purchase behavior using personalized promotions and deals. (Dormehl, 2014).

”The problem with the web and its associated technologies is that it has made it so easy to share information about ourselves that doing so begins to feel like an obligation, a sentence, a sisyphean curse; another day, another photo of lunch to post.”

Facebook manipulates our emotions with experiments on the News Feed. (Booth, 2014).

Facebook owns a patent for determining our location by identifying objects in our photos and videos based on neural networks of nearby images. (Facebook, 2014).

Facebook payments is tracking what you buy, and your financial information like bank account and credit card numbers. Facebook has already started sharing data with Mastercard so they can drive online ad sales and determine credit worthiness from platform data. (Head, 2014).

Facebook provided data to NSA as part of the PRISM program. (Greenwald, 2013).

Facebook enables a surveillance apparatus where our every action could be monitored, and since everyone technically violates some obscure law some of the time, then punishment becomes selective and political. (Marlinspike, 2013).

Facebook owns a patent that tracks and determines the types of physical activities of a user based on movement of their device, including walking, running, cycling, driving, skiing, etc. (Facebook, 2013).

Facebook owns a patent for determining our location using non-GPS data including nearby NFC, RFID, wifi, bluetooth signal, events in your calendar like restaurant reservations or concerts. (Facebook, March 2013)

Facebook sells profile data to credit card companies and insurance providers so they can use platform data as indicators for credit and insurance risk. (Hawley, 2012).

Facebook makes it very difficult to quit, using social reciprocity and UX design dark patterns. (Brown, 2010).

Facebook’s CEO Mark Zuckerberg hacked into Harvard Crimson editors’ private email accounts. (Carlson, 2010).

You should delete your Facebook account, but please share this essay before you do 😉!

This is part of a series exploring idleness, time, attention, and lived experience.

1. Travel Writing - Redux, Italy & Switzerland Edition
2. Against Facebook
3. I am not a software developer
4. The Disappearance of Lived Time

Bibliography

Angwin, Julia. Mattu, Surya. Parris Jr, Terry. (December 27, 2016). Facebook Doesn’t Tell Users Everything It Really Knows About Them (Retrieved April 21, 2018).

Ahmed, Nafeez. (December 29, 2017). Facebook will become more powerful than the NSA in less than 10 years — unless we stop it. (Retrieved April 10, 2018).

Allen, Tom. (2016). How my Location Independent Lifestyle Works. (Retrieved on April 19, 2018)

Allsop, John. (February 17, 2017). Not My Digital Detox. (Retrieved April 19, 2018)

Ashkenas, Jeremy. (April 4, 2018). “You know, I really hate to keep beating a downed zuckerberg, but to the extent that expensive patents indicate corporate intent and direction —Come along for a ride, and let’s browse a few of Facebook’s recent U.S.P.T.O. patent applications…“. Twitter. https://mobile.twitter.com/jashkenas/status/981672970098589696 (Retrieved on April 19, 2018)

Assange, Julian., Appelbaum, Jacob., Müller-Maguhn, Andy., Zimmermann, Jérémie. (2012). Cypherpunks: Freedom and the Future of the Internet. OR Books. Print.

Balkan, Aral. (February 18, 2017). Encouraging individual sovereignty and a healthy commons. (Retrieved on January 13, 2018).

Beato, G. (March 2012). Disposable Hip. (Retrieved April 27, 2018).

Berthelot, Martin R. (2013). Spectacle and Resistance in the Modern and Postmodern Eras. (Retrieved on April 18, 2018)

Berthelot, Martin R. (2013). Spectacle & Resistance. (Retrieved on April 18, 2018)

Beller, Jonathan. (2006). The cinematic mode of production: attention economy and the society of the spectacle. Hanover, N.H. Dartmouth College Press, University Press of New England. Print.

Booth, Robert. (January 29, 2014). Facebook Reveals News Feed Experiment to Control Emotions. (Retrieved April 19, 2018)

Boykis, Vicki. (February 1, 2017). What should you think about when using Facebook?. (Retrieved on January 13, 2018).

Brand, Russell., Curtis, Adam. (2018). Ep.50 “Is Civilisation Crumbling? with Adam Curtis” by Under The Skin Podcast with Russell Brand. (Retrieved on March 17, 2018).

Bridle, James. (November 6, 2017). Something is wrong on the internet. (Retrieved November 8, 2018).

Brown, Andrew. (May 14, 2010). Facebook is not your friend. (Retrieved on June 5, 2016).

Burrington, Ingrid. (December 4, 2015). A Journey Into the Heart of Facebook. (Retrieved on January 3, 2016).

Cadwalladr, Carole, Graham-Harrison, Emma. (March 17, 2018). How Cambridge Analytica turned Facebook ‘likes’ into a lucrative political tool. (Retrieved on April 10, 2018).

Cadwalladr, Carole, Graham-Harrison, Emma. (March 17, 2018). Revealed: 50 million Facebook profiles harvested for Cambridge Analytica in major data breach. (Retrieved on April 10, 2018).

Carlson, Nicholas. (March 5, 2010). In 2004, Mark Zuckerberg Broke into a Facebook User’s Private Email Account. (Retrieved April 21, 2018).

CB insights. (June 1, 2017). Facebook’s Emotion Tech: Patents Show New Ways For Detecting And Responding To Users’ Feelings. (Retrieved on June 5, 2017).

Clark, Bryan. (February 11, 2016). ‘I have nothing to hide’ is killing the privacy argument. (Retrieved on January 13, 2017).

Crary, Jonathan. (2001). Suspensions of perception: attention, spectacle, and modern culture. London, England. MIT Press.

Debord, Guy. (1983). The Society of the Spectacle. Trans. Fredy Perlman. Detroit: Black & Red, Print.

Derakhshan, Hossein. (December 29, 2015). Iran’s blogfather: Facebook, Instagram and Twitter are killing the web. (Retrieved on January 13, 2018).

Derakhshan, Hossein. (July 14, 2015). The Web We Have to Save. (Retrieved on January 13, 2018).

Derakhshan, Hossein. (May 11, 2016). Mark Zuckerberg is a hypocrite - Facebook has destroyed the open web. (Retrieved on January 13, 2018).

Dillet, Romain. (October 20, 2017). How I cured my tech fatigue by ditching feeds. (Retrieved on April 10, 2018)

Dobelli, Rolf. (2020). Stop Reading the News: A Manifesto for a Happier, Calmer and Wiser Life. Hachette UK.

Dormehl, Luke. (2014). The Formula. New York: Perigee. Print.

Dowling, Tim. (June 6, 2017). On Facebook, even Harvard students can’t be too paranoid. (Retrieved on June 5, 2016).

Dwoskin, Elizabeth. (April 30, 2018). WhatsApp founder plans to leave after broad clashes with parent Facebook (Retrieved on April 30, 2018).

Ekbia, Hamid., Nardi, Bonnie. (June 2, 2014). Heteromation and its (dis)contents: The invisible division of labor between humans and machines. (Retrieved on April 23, 2018).

Gruber, John. (June 6, 2017). Fuck Facebook. (Retrieved on June 6, 2017).

Facebook, Inc. (2017). Facebook 2017 Annual Report, Form 10-K. US Securities and Exchange Commission. (Retrieved on April 19, 2018)

Facebook, Inc. (June 28, 2013). User Activity Tracking System. US Patent Office. (Retrieved April 24, 2018).

Facebook, Inc. (December 30, 2014). Systems and methods for image object recognition based on location information and object categories. US Patent Office. (Retrieved April 24, 2018).

Facebook, Inc. (March 15, 2013). Multi-Factor Location Verification. US Patent Office. (Retrieved April 24, 2018).

Faliszek, Chet. (April 2, 2018). “I recently posted about Oculus/Facebook and their data collection. Let me go more in depth and this isn’t just about today this is about the future of XR. At the heart of the matter are these points where their privacy policy and actions differ from other XR companies. 1/many”. Twitter. https://twitter.com/chetfaliszek/status/980861065989783552. (Retrieved on April 19, 2018)

Ferri, Jessica. (2018). How Swipe Left, Swipe Right Became a Cultural Phenomenon. (Retrieved on April 19, 2018)

Frost, Brad. (September 11, 2017). Facebook, You Needy sonofabitch. (Retrieved April 21, 2018).

Greenwald, Glenn, MacAskill, Ewen. (June 7, 2013). NSA Prism Program Taps in to User Data of Apple, Google and others. (Retrieved April 18, 2018)

Harris, Tristan. (May 18, 2016). How Technology is Hijacking Your Mind — from a Magician and Google Design Ethicist. (Retrieved on April 9, 2018).

Hawley, Charles. (June 8, 2012). Critique of German Credit Agency Plan to Mine Facebook for Data. (Retrieved April 21, 2018).

Head, Beverley. (October 6, 2014). MasterCard to Access Facebook User Data. (Retrieved April 21, 2018).

Hern, Alex. (October 25, 2017). ‘Downright Orwellian’: journalists decry Facebook experiment’s impact on democracy. (Retrieved on April 10, 2018).

Hern, Alex. (January 23, 2018). ‘Never get high on your own supply’ – why social media bosses don’t use social media. (Retrieved on April 10, 2018).

Hess, Amanda. (May 9, 2017). How Privacy Became a Commodity for the Rich and Powerful. (Retrieved on November 3, 2017).

Hill, Kashmir. (July 11, 2017). How Facebook Figures Out Everyone You’ve Ever Met. (Retrieved on April 10, 2018).

Kaplan, Frederic. (August 1, 2014). Linguistic Capitalism and Algorithmic Mediation. University of California Press Journals. (Retrieved April 18, 2018)

Kalish, Alyse. (February 7, 2018). 15 Things you should be doing after work instead of watching TV. (Retrieved on April 19, 2018)

Krause, Kati. (December 11, 2015). Facebook’s Mental Health Problem. (Retrieved on January 3, 2016).

Lanchester, John. (August 17, 2017). You Are the Product. (Retrieved on April 9, 2018).

Laney, Doug. (May 3, 2012). To Facebook You’re Worth $80.95. (Retrieved April 24, 2018).

Lewis, Paul. (October 6, 2017). ‘Our minds can be hijacked’: the tech insiders who fear a smartphone dystopia. (Retrieved on April 9, 2018).

Lomas, Natasha. (February 19, 2018). Facebook’s tracking of non-users ruled illegal again. (Retrieved on April 10, 2018).

Machkovech, Sam. (May 1, 2017). Report: Facebook helped advertisers target teens who feel “worthless”. (Retrieved on November 3, 2017).

Malik, Om. (February 20, 2018). The #1 Reason Facebook Won’t Ever Change. (Retrieved April 19, 2018)

Marlinspike, Moxie. (June 13, 2013). Why ‘I Have Nothing to Hide’ Is the Wrong Way to Think About Surveillance. (Retrieved on January 3, 2017).

McMillen, Stuart. (March 2012). Amusing Ourselves to Death. (Retrieved on April 30, 2018).

Miller, Joe. (May 26, 2016). How Facebook’s tentacles reach further than you think. (Retrieved on August 1, 2017).

Musser, Cody. (December 28, 2015). I’m quitting Facebook in 2016—and you should too. (Retrieved on January 13, 2018).

NBC News. (April 10, 2018). Senator Asks How Facebook Remains Free, Mark Zuckerberg Smirks: ‘We Run Ads’ | NBC News (Retrieved April 23, 2018).

Nelson, Joe. (April 20, 2015). Going “Write-Only”. (Retrieved April 20, 2018)

Newton, Casey. (February 6, 2018). Facebook hired a full-time pollster to monitor Zuckerberg’s approval ratings. (Retrieved on April 10, 2018).

Newton, Casey. (March 20, 2018). WhatsApp co-founder tells everyone to delete Facebook. (Retrieved on April 30, 2018).

Nunez, Michael. (May 9, 2016). Former Facebook Workers: We Routinely Suppressed Conservative News. (Retrieved on November 3, 2017).

Nunez, Michael. (May 3, 2016). Want to Know What Facebook Really Thinks of Journalists? Here’s What Happened When It Hired Some. (Retrieved on November 3, 2017).

Parakilas, Sandy. (November 19, 2017). We Can’t Trust Facebook to Regulate Itself. (Retrieved on April 18, 2018).

Parez, Sarah. (February 12, 2018). Facebook is pushing its data-tracking Onavo VPN within its main mobile app. (Retrieved on April 10, 2018).

Parrish, Shane. (January 24, 2018). Most of what you’re going to read today is pointless. (Retrieved on April 10, 2018).

Perlroth, Nicole, Frenkel, Sheera & Shane, Scott. (March 19, 2018). Facebook Exit Hints at Dissent on Handling of Russian Trolls. (Retrieved on April 10, 2018).

Reynolds, Emily. (November 1, 2016). What could Facebook target next? Our mental health data. (Retrieved on November 1, 2016).

Russell, Bertrand. (2013). The Conquest of Happiness. Liveright, Print.

Satyal, Parimal. (November 2, 2017). Against an Increasingly User-Hostile Web. (Retrieved on April 10, 2018).

Solnit, Rebecca. (May 1, 2018). Driven to Distraction. (Retrieved April 23, 2018).

Solon, Olivia. (June 16, 2017). Revealed: Facebook exposed identities of moderators to suspected terrorists. (Retrieved on April 9, 2018).

Smith, Jack IV. (February 24, 2016). Facebook Is Using Those New “Like” Emojis to Keep Tabs on Your Emotions. (Retrieved on April 20, 2018)

Srnicek, Nick. (August 30, 2017). We need to nationalise Google, Facebook and Amazon. Here’s why. (Retrieved on April 9, 2018).

Stallman, Richard. (2016). Reasons not to use Facebook. (Retrieved on June 5, 2016).

Staltz, André. (October 30, 2017). The Web began dying in 2014, here’s how. (Retrieved on April 10, 2018).

Stern, Joanna. (March 7, 2018). Facebook Really Is Spying on You, Just Not Through Your Phone’s Mic. (Retrieved on April 10, 2018).

Swartz, Aaron. (October 20, 2006). I Hate The News. (Retrieved May 2, 2018).

Taleb, Nassim Nicholas. (2012). Antifragile: Things That Gain from Disorder. Random House. Print.

Tanner, Grafton. (August 9, 2018). Digital Detox: Big Tech’s Phony Crisis of Conscience. (Retrieved on March 3, 2021)

Thaler, Richard H., and Cass R. Sunstein. (2008). Nudge: Improving Decisions About Health, Wealth, and Happiness. New Haven: Yale University Press. Print.

Turner, Fred. (October 5, 2017). The arts at Facebook: An aesthetic infrastructure for surveillance capitalism. (Retrieved on April 21, 2018).

Thoreau, Henry David. (1863). Life Without Principle. (Retrieved on April 19, 2018)

Tucker, Ian. (February 12, 2016). Douglas Rushkoff: ‘I’m thinking it may be good to be off social media altogether’. (Retrieved on January 13, 2018).

Virani, Salim. (January 29, 2015). Get your loved ones off Facebook. (Retrieved on April 9, 2018).

Wald, Chelsea. (December 3, 2015). Is Facebook Luring You Into Being Depressed?. (Retrieved on January 3, 2016).

Wolford, Ben. (October 23, 2017). The product Facebook sells is you. (Retrieved on January 3, 2016).

Zuboff, Shoshana. (May 3, 2016) The Secrets of Surveillance Capitalism. (Retrieved on January 3, 2017).

Ember-changeset is an addon that provides an abstraction over Ember models such that only valid data will ever be set in a model.

Contextual components are a set of conventions that allow a developer to build a component that can be customized by its consuming application. This means they can both yield values to its consumer, and allows a consumer to customize the display of the component.

Ch.1

The layers of phoenix, the endpoint is where the HTTP connection contacts phoenix, from there it goes to the router which directs a request to the appropriate controller, passing through a series of pipelines. Pipelines chain functions together to handle tasks that span across multiple controllers, such as browser requests vs API requests.

connection
|> endpoint
|> router
|> pipelines
|> controller

The purpose of the web server is to route a request to a function that performs the right task, called an action. These functions are grouped into controllers. A controller is also a series of pipelines

connection
|> controller
|> common_services
|> action

An action to show a user would look like this:

connection
|> find_user
|> view
|> template

Run phoenix inside an interactive shell: iex -S mix phx.server

In controllers, parameters come in as a Map of keys with string names:

```
def world(conn, %{"name" => name}) do    # string key
  render conn, "world.html", name: name  # atom key
end
```

Phoenix specifically converts these from String keys to atoms once inside the application boundry, because external data can’t safely be converted to atoms.

Phoenix encourages breaking big functions down into smaller ones. It provides a place to explicitly register each smaller function in a way that makes it easy to understand and replace. These small pieces are orchestrated with the Plug library. The Plug library is a specification for building applications that connect with the web. Plug produces and consumes a struct called Plug.Conn, describing the entire request/response cycle of a web application, containing the request, protocol, parameters, etc. Each individual Plug should do one-thing and do it well, take the conn struct, transform it, and return it.

Plugs are just functions, and Phoenix web applications are just pipelines of Plugs.

If you look at endpoint.ex you’ll see that its essentially a pipeline of functions chained together at the beginning of each request:

connection
|> Plug.Static.call
|> Plug.RequestId.call
|> Plug.Logger.call
|> Plug.Parsers.call
|> Plug.MethodOverride.call
|> Plug.Head.call
|> Plug.Session.call
|> <your app>.Router.call

The last call, to <your app>.Router.call is where endpoint.ex defers to your applications router.ex to define which controller, and subsequently, which function is invoked next in the pipeline chain. The router defines a set of pipelines for dealing with which Plugs to use prior to your conroller being called, typically:

connection
|> endpoint
|> router
|> pipeline(s)
|> controller

The pipeline(s) above will be things like :browser, handling common tasks as preventing request forgery, :fetch_session, :fetch_flash.

The controller is where business logic specific to your application lives, making data available for a view. The view subsequently substitutes values inside a template which is rendered.

Ch.2

Start an interactive Elixir shell within an Elixir application project:

iex -S mix

Structs add a constraint to Maps in that they offer protection from bad or misspelled keynames at compilation time, while Maps only throw errors during runtime when the key is first accessed. Under the covers, a Struct is just a Map with a struct key.

Ch3

A view is a module containing a rendering function that converts data into a format the user will consume, like HTML or JSON. A template is a function on the view module, compiled from a file containing raw markup language and embedded Elixir code to process substitutions and loops. Views are modules responsible for rendering, and templates are the compiled functions that have static markup and native code to build pages.

EEx executes Elixir code within <%= %> tags, injecting the result into the template. EEx evalutes code within <% %> tags, but without injecting the result.

The expression <%= for user <- @users do %> walks through the users, rendering each user using the template block between the do ... end block, rolling the result up into the template.

In Phoenix, after compilation, templates are just functions. These templates are just linked lists (rather than other web frameworks that use string concatenation)– this means Phoenix doesn’t have to make huge copies of large strings, resulting in performance improvements. This lets Elixir leverage modern CPU optimization since it only has a single copy of the largest strings in your application.

When a template is added to Phoenix, the view module extracts the template from the filesystem and adds it as a function on the module itself, thats why the view module is required in the first place.

Ch.4

Ecto uses a DSL that specifies how the fields in a struct map to database tables and columns. The DSL is built using Elixir macros. Virtual fields are not persisted to the database.

Models (called contexts & schemas in Phoenix 1.3) are layers of functions to transform data according to business logic. A schema is the native database form of the data, and a struct is the data once pulled out of the database. Changesets in Ecto decouple the authentication update policy separately from the database schema. This policy segregation makes handling change over time.

Ecto helps you write code to make the minimal required database operation to update a record. Ecto can enforce constraints without hitting the database.

Ch.5

Ecto.Changeset.cast converts a bare Map to a changeset, for security it limits the incoming parameters to those specified.

Module plugs vs function plugs: A function plug is a single function, a module plug is a module that provides two functions with some configuration details, otherwise they work the same. lib/rumbl_web/endpoint.ex is an example of a module plug while lib/rumbl_web/router.ex is an example of a function plug. You specify a module plug by providing a module name plug Plug.Logger while a function plug is specified by its name as an atom plug :protect_from_forgery.

To share a Plug across more than one module, you want to use a Module Plug. All plugs take a connection and return a connection.

Any values stored in conn.assigns.* is automatically available with the same name in our view. So if you assign to current_user, then @current_user can be used directly in a view.

Ch.6 Generators & Relationships

Generating resources, phx.gen.html generates simple HTML based scaffolds, phx.gen.json generates a REST-based API using JSON. They create resources for CRUD operations, creating migrations, controllers, and templates as well as tests.

An example:

mix phx.gen.html Medias \
  Video videos \
  user_id:references:users \
  url:string \
  title:string \
  description:text

This includes the name of the context that holds the module, the module defines the model. The plural form of the model name, and each field with some type information. Phoenix consistently uses singular forms in models, controllers, and views. At the application boundry (URLs, table names you provide a bit more information because you can use pluralized names). Instead of using inflectors, you just explicitly set the pluralized form yourself.

Primary keys identify rows for each item in a table. Foreign keys point from one table to the primary key in another table. Foreign key lets the database get in on the act of maintaining consistency across our two relationships.

The change function handles two database functions, one for migrating up and one for migrating down. A migration up applies a migration, and a migration down reverts it. If you make a mistake a need to move a single migration up or down, you can do so.

If you meant to add a view_count field to your create_video migration, but before you migrated your database up. You would create a new migration adding the view_count field, since you haven’t pushed your changes to production yet, you can roll back, make your changes, and then migrate up again. First you’d roll back your changes: mix ecto.rollback Verify the undo of the create_video migration. At this point, add the missing view_count field and migrate forwards with mix ecto.migrate.

A schema is responsible for tying the database fields to a field in the Elixir struct, these are defined in lib/<appname>/<contextname>/<model>.ex.

Ecto associations are explicit, you need to ask to fetch associated records specifically. Most persistence frameworks often fetch rows you don’t need or fetch them in inefficient ways, over time these inefficiencies add up to major performance problems. Ecto forces the developer to pay down these performance hits early so they don’t add up over time, when they’re more difficult to fix.

iex> user = Repo.get_by!(User, username: "josevalim")
%{Rumbl.User{...}
iex> user.videos
#Ecto.Association.NotLoaded<association :videos is not loaded>
iex> user = Repo.preload(user, :videos)
%{Rumbl.User{...}
iex> user.videos
[] # loaded, but none are associated! yay

Repo.preload accepts one or a collection of association names, and fetches all associated data.

iex> user = Repo.get_by!(User, username: "josevalim")
iex> attrs = %{title: "hi", description: "says hi", url: "example.com"}
iex> video = Ecto.build_assoc(user, :videos, attrs)
iex> video = Repo.insert!(video)

Ecto.build_assoc builds a struct, with the proper relationship fields already set.

To fetch videos associated with a user, without storing them in a user struct:

iex> query = Ecto.assoc(user, :videos)
iex> Repo.all(query)
[%Rumbl.Video{...}]

assoc is convenient in that it returns an Ecto.Query all videos scoped to a specific user, or a list of users, we convert this query into data by calling Repo.all.

Ch.7 Ecto Queries and Constraints

Seeds are small scripts that populate the database with values every time the script is run. Phoenix stores seed scripts in priv/repo/seeds.ex. Mix tasks will run these scripts. Use mix run priv/repo/seeds.exs to add the seed data to the database.

Repo.all from c in Category, select: c.name

Repo.all means return all rows, from is a macro that builds a query, c in Category means we’re pulling rows (labeled c) from the Category schema. select: c.name means we’re going to return only the name field.

Repo.all from c in Category, order_by: c.name, select: c.name

will order the results by name and return a tuple containing the name and the id fields.

Ecto queries are composable, you don’t need to define the entire query at once, you can combine them bit-by-bit.

iex> query = Category
iex> query = from c in query, order_by: c.name
iex> query = from c in query, select: {c.name, c.id}
#Ecto.Query<>
iex> Repo.all(query)
[...]

This works because Ecto defines a queryable protocol. from receives a queryable, and you can use any queryable as a base for a new query. An Elixir protocol defines an API for specific language features, this one defines the API for something that can be queried. This is why we can invoke Repo.all(Category) or Repo.all(query) because both Category and query implement the Ecto.Queryable protocol. By abiding to the protocol, developer can quickly layer together sophisticaed queries with Ecto.Query, maintaining sophistication without complexity.

Code that builds and transforms queries, and code that interacts with the repository should belong to the context. Code that makes requests for the data should belong to the controller—because the controller is where the web logic should live, and the database layer should be hidden within the application context with the rest of the business logic.

iex> username = "josevalim"
iex> Repo.one(from u in User, where: u.username == ^username)
%Rumbl.User{...}

Repo.one returns one row, from u in user means read from the User schema. where u.username == ^username means return the row where u.username == ^username, using the pin ^ operator means we don’t want to assign the username but use its value.

Repo.one does not mean “return the first result” but “one result is expected”, so if there is more, it fails. The Ecto Query API is not about composing query strings, it uses Elixir macros such that Ecto knows where user-defined variables are located, it’s easier to protect the user from security flaws like SQL-injection attacks. It also helps a bit with query normalization and leverages the data types as defined in the schema for casting values at runtime.

Any functions with side effects—the ones that change the world—should remain in the controllers, while the context, model, and view layers remain side effect free. The controller receives data, either from a traditional web request, reading data from a socket, and this data is passed from the controller to various functions that transform it as it moves through the functions to the shape of our business-model requirements. Finally it makes changes to the world around us, either delivering emails, adding entries to a database, or invoking a view which is again written to the connection (another side effect), any of which can result in a business operation.

The query API supports many operators: ==, !=, <=, >=, <, >, and, or, not, in, like, ilike, is_nil, count, avg, sum, min, max, datetime_add, date_add, fragment, field, type.

Keyword syntax uses a keyword list to express a query.

iex> Repo.one from u in User,
              select: count(u.id),
              where: ilike(u.username, ^"j%") or
                     ilike(u.username, ^"c%")

the u variable is bound as part of Ecto’s from macro, representing entries from the User schema. Each join in a query gets a special binding.

# count users
iex> users_count = from u in User, select: count(u.id)
#Ecto.Query<from u in Rumbl.User, select: count(u.id)>
# count usernames with a j
iex> j_users = from u in users_count, where: ilike(u.username, ^"%j%")
#Ecto.Query<from u in Rumbl.User, where: ilike(u.username, ^"%j%"), select: count(u.id)>

This query builds up a new query, normalizing as it builds, upon the saved query, we even built the query using the same bound variable name, u, but we didn’t have to.

The pipe syntax allows developer to build queries by piping through query macros. Each pipe takes a queryable and returns a queryable.

iex> User |>
     select([u], count(u.id)) |>
     where([u], ilike(u.username, ^"j%") or ilike(u.username, ^"c%")) |>
     Repo.one()
[debug] QUERY OK source="users" db=4.5ms
SELECT count(u0."id") FROM "users" AS u0 WHERE ((u0."username" ILIKE $1) OR (u0."username" ILIKE $2)) ["j%", "c%"]
5

Because each segment of the pipe works independently of the others, we need to specify the binding manually for each one.

Fragments offer an escape hatch from Ecto’s Query API. The best abstractions offer an escape hatch, and since Ecto’s Query API doesn’t represent every query the database layer can provide, Ecto’s query fragments send part of the query directly to the database but allows you to construct it in a safe way, like this: iex> from(u in User, where: fragment(“lower(username) = ?”, ^String.downcase(uname)))

When all else fails, you can directly run SQL statements with Ecto.Adapters.SQL.Query:

iex> Ecto.Adapters.SQL.query(Rumbl.Repo, "SELECT power($1, $2)", [2, 10])
SELECT power($1, $2) [2, 10]
{:ok, %Postgrex.Result{ columns: ["power"], command: :select, connection_id: 5979, num_rows: 1, rows: [[1024.0]] }}

Ecto relationships are explicit:

iex> user = Repo.one from(u in User, limit: 1)
iex> user.videos
#Ecto.Association.NotLoaded<association :videos is not loaded>
iex> user = Repo.preload(user, :videos)
iex> []

Ecto allows us to preload associations directly as part of a query:

iex> Repo.all from u in User, join: v in assoc(u, :videos), join: c in
assoc(v, :category), where: c.name == "Comedy", select: {u, v}
[{Rumbl.User{...}, %Rumbl.User{...}}]

Constraints allow developers to use underlying relational, and can solve potential race conditions:

1. User sends a category ID through the form.
2. Perform a query to check if the category ID exists in the DB.
3. If the category ID exists, add the video with the category ID to the database.

However, someone could delete the category ID between steps 2 and 3, allowing a video insertion without and existing category. This will lead to inconsistent data over time. Phoenix uses constrains to manage change in a way that combines the harsh protections of the database with Ecto’s gentle guiding hand to report errors without crashing. Some terms:

constraint: An explicit database constraint, a uniqueness constrain, or an index, or and integrity constraint between primary and foreign keys.

constraint error: The Ecto.ConstraintError.
changeset constraint: A constraint annotation allowing Ecto to convert constraint errors into changeset error messages.
changeset error messages: Beautiful error messages for the consumption of people.

Ecto allows the application layer (and web server) to use database services like referential integrity and transactions to strike a balance between the needs of the application layer and the needs of the database. Ecto rewards developers the many guarantees databases offer with data integrity in the short term, by transforming constraint errors into user feedback, and in the longer term by guaranteeing you wont be awake at 3am fixing bugs caused by inconsistent data.

IEx allows us to fetch a previous value using v(n), where n is the number of the expression, you can alsop ass negative values to grab the last nth expression.

iex> alias Rumbl.Repo
Rumbl.Repo
iex> category = Repo.get_by(Rumbl.Medias.Category, name: "Drama")
%Rumbl.Medias.Category{
  __meta__: #Ecto.Schema.Metadata<:loaded, "categories">,
  id: 2,
  inserted_at: ~N[2017-11-05 18:02:12.006642],
  name: "Drama",
  updated_at: ~N[2017-11-05 18:02:12.006653]
}
iex> Repo.delete(category)
[debug] QUERY ERROR db=6.5ms
DELETE FROM "categories" WHERE "id" = $1 [2]
** (Ecto.ConstraintError) constraint error when attempting to delete struct: * foreign_key: videos_category_id_fkey

We previously added a video using the Drama category, so the database prevents deletion of the category as it is tied to the video and would create orphaned records. Use foreign_key_constraint, which is like assoc_constraint used earlier, except it doesn’t inflect the foreign key from the relationship.

This is particularly useful if you want to provide reasons why a category can not be deleted.

iex> import Ecto.Changeset
Ecto.Changeset
iex(22)> changeset = Ecto.Changeset.change(category)
#Ecto.Changeset<action: nil, changes: %{}, errors: [], data: #Rumbl.Medias.Category<>, valid?: true>
iex> changeset = foreign_key_constraint(changeset, :videos, name: :videos_category_id_fkey, message: "still exists")
#Ecto.Changeset<action: nil, changes: %{}, errors: [],
            data: #Rumbl.Medias.Category<>, valid?: true>
iex> Repo.delete(changeset)
[debug] QUERY ERROR db=8.8ms
DELETE FROM "categories" WHERE "id" = $1 [2]
{
  :error,
  #Ecto.Changeset<
    action: :delete,
    changes: %{},
    errors: [videos: {"still exists", []}],
    data: #Rumbl.Medias.Category<>,
    valid?: false
  >
}

because the foreign key is established in the videos table, we need to explicitly call this out in the call to foreign_key_constraint. The work best suited for the database must be done in the database.

The constraint changeset functions are useful when the constraint being mapped is triggered by external data, often as part of the user request. Using changeset constraints only makes sense if the error message can be something the user can take useful action on.

Ch.8 Testing MVC

Testing principals

• Fast — tests should run quickly and concurrently if possible
• Isolated — Tests that are too isolated wont have enough context to be useful, tests that aren’t isolated enough are difficult to understand and maintain
• DRY — Strive to eliminate repetition in tests
• Repeatable — The same test on the same code should always yield the same result
• unit test — exercises a function for one layer of your application
• integration test — focuses on the way different layers of an application fit together. An example is a controller test that makes a request to and endpoint, flows through the pipelines, reads from the database, and renders through views just like a Phoenix request would
• acceptance test — test how multiple actions work together. A single acceptance test case may sign the user on, perform several calculations that build upon each other, then sign off.

ExUnit has three main macros, setup macro specifies setup code that runs before each test is run. test macro defines a single isolated test, it defines a hypothesis about code. Finally, assert macro specifies something we believe to be true about code, if true, the test passes.

defmodule MyTest do
  use ExUnit.Case, async: true
  setup do
    :ok
  end
  test "pass" do
    assert true
  end
  test "fail" do
    assert false
  end
end

This code runs two tests, first setup is run, then the “pass” test case. Next setup is run, and the “fail” test case. The output is a passing test case and a failing test case.

Tests will generally use <AppName>.ConnCase meaning you get <AppName>.Router.Helpers, and Ecto imports along for free.

ExUnit calls tests with

test "GET /", %{conn: conn} do
  conn = get(conn, "/")
  assert html_response(conn, 200) =~ "Welcome to Phoenix!"
end

by calling get(conn, ”/”) rather than calling the index action on the controller directly. This practice gives the test the right level of isolation because we’re using the controller the same way Phoenix does. Phoenix also provides test helpers to make testing responses easier.

assert html_response(conn, 200) ~= "Welcome to Phoenix!"

which does the following:

asserts the response was 200
content-type was text/html
return the response body, allowing us to match on the contents

this also exists for JSON called json_response that allow for assertions like

assert %{user_id: user.id} = json_response(conn, 200)

ExUnit tags help when setup is different from test to test. When specifying a tag, ExUnit makes that information available within the setup block via callbacks.

setup %{conn: conn} = config do
  if username = config[:login_as] do
    user = insert_user(username: username)
    conn = assign(conn, :current_user, user)
    {:ok, conn: conn, user: user}
  else
    :ok
  end
end
@tag login_as: "max" # this is the tag
test "list all user's videos on index", %{conn: conn, user: user} do
# ...

tags take either a keyword list (above) or an atom as arguments. The config map contains the conn and tag (either atom or keyword list). In this case if login_as has a value, we use it to login the user and return an updated conn with the user, otherwise we return an :ok.

Writing unit tests directly against a function like the Auth plug will result in unexpected errors during the testing/coding cycle because calls directly to the function will get results that have not gone through the endpoint, router or other pipelines. The data has not been fully transformed, so the results will not reflect the state that the implementation sees when running in a server. The bypass_through helper that ConnCase provides allows developer to send a connection through the endpoint, router, and desired pipelines but bypass the route dispatch. This approach gives developer a connection wired up with all the transformations the specific tests require, such as fetching the session and adding flash messages.

setup %{conn: conn} do
  conn =
    conn
    |> bypass_through(RumblWeb.Router, :browser)
    |> get("/")

  {:ok, %{conn, conn}}
end
test "authenticate_user halts when no current_user exists", %{conn: conn} do
  conn = Auth.authenticate_user([])
  assert conn.halted
end

The setup block calls bypass_through, passing the router and the :browser pipeline to invoke. When the get request is invoked, it accesses the endpoint and stops at the browser pipeline, as requested. The path ”/” given to the get isn’t used by the router when bypassing, its just stored in the connection, this provides all the requirements for a plug with a valid session and flash message support.

Since most repository-related functionality will be tested with integration tests as they insert and update records, but we want to be sure to catch some error conditions as close to the breaking point as possible. One example is the uniqueness constraint checks in the changeset. It has side effects because we’re going to need to create a record and then test against it.

Ch. 9: Watching Videos

alter table(:videos) do
  add :slug, :string
end

the alter macro changes the schema for both up and down migrations. The premise of a slug is so you can permanently generate a name from the data in other fields, some of which may be updatable.

Because Ecto separates changesets from the definition of a database record (model), this gives developer the ability to have different change policies for any type of change to the record. Changesets act as a funnel to the database, filtering and casting incoming data, making sure sensitive fields like user role cannot be set externally, while casting them to the type defined in the schema. Changesets can validate data, for example the length of a field— on the fly, but validations that depend on database integrity are left to the database, in the form of constraints. Changesets make the code easier to reason about, can be composed, allowing each part of a data transformation to be a more easily comprehend–able and testable function.

In viws you may see a video_path, or watch_path, or x_path() helpers. These helper functions are generated by RumblWeb.Router and imported into controllers and views by rumble_web/web.ex. watch_path(@conn, :show, video) takes the Video struct, and by conventions it generates a URL by extracting the ID from the Video struct. This works because Phoenix knows to use the ID field to generated the URL because Phoenix implements the Phoenix.Param protocol. Developer can customize this behavior by implementing our own functions to override the default protocol implementation to generate the URL using the slug.

defimpl Phoenix.Param, for: Rumbl.video do
  def to_param(%{slug: slug, id: id}) do
    "#{id}-#{slug}"
  end
end

The advantage of implementing this behavior change as a Protocol implementation is the clean polymorphism we get by extending Phoenix parameters without having to change Phoenix itself or the Video module itself.

Ch. 10: Using Channels

A client connects to a channel over a socket, then sends and receives messages, that’s it. A Phoenix channel is a conversation, the channel sends messages, receives messages, and keeps state. Just as controllers receive requests, channels receive events.

A Phoenix conversation (messages sent, messages received) is about a topic, and the topic maps onto application concepts like a chat room, local map, or annotations to a video. Each user’s conversation on a topic has its own isolated, dedicated process.

Whereas the traditional web-based request/response interactions are stateless, conversations in an Elixir process can be stateful. This means for sophisticated interactions like games and more interactive pages, developer doesn’t need to use local storage, cookies, or databases to keep track of the state of the conversation. Each call to the channel simply picks up where it left off. This only works if the framework supports true isolation and concurrency: one process must be isolated, not affect others, and true concurrency means lightweight abstractions that don’t bleed into one another. Channels must scale in the dimensions developer is most concerned about: code complexity, performance, and manageability.

The application channel is simple, but it must manage three things:

making and breaking connections
sending messages
receiving messages

A socket establishes a new connection with a socket. After a connection is made, that same socket will be transformed through the life of the connection. The socket is a representation of a continuous ongoing conversation between the client and the server. When a connection is made, the initial socket is created, and that same socket is transformed with each new received event, through the whole life of the conversation.

The socket is an abstraction over a websocket, or to handle older clients, a longpoll. The socket abstracts the transport layer, and Phoenix handles the rest.

A channel is a conversation about a topic, the topic has an identifier of videos:<video id> where video_id is a dynamic id matching a record in the database. Topic identifiers generally take the form topic:subtopic where topic is often a resource name and subtopic is the instance ID that usually identifies a row in the database.

Topics are organizing concepts, they are used as parameters to functions, and used in URLs to identify conversations. Just like a URL passes an :id parameter to represent a request for a resource from a controller; with channels, developer can use the topic ID to scope a channel connection.

Phoenix channels define “callbacks”, or functions that respond to channel events such as join below:

def join("videos:" <> video_id, _params, socket) do
  {:ok, assign(socket, :video_id, String.to_integer(video_id))}
end

in the above code, assign(socket…) returns a socket, resulting in a {:ok, socket} returning for authorized users or {:error, socket} which will deny joining the channel. Sockets maintain state in a socket.assigns map, and the above adds the video id to the assigns map using the helper function assign(socket, :video_id, String.to_integer(video_id)).

Socket state is maintained for the duration of the connection. The socket is transformed in a loop rather than a single pipeline. The socket state changed in the above “assign()” call will be accessible later as events come into and out of the channel. This small difference leads to enormous differences in efficiency between the channels API and the controllers API.

Channels receive events with an event name, such as new_message, and include a payload of arbitrary data. Each channel module can receive events in three ways

handle_in receives direct channel events
handle_out intercepts broadcast events
handle_info receives OTP messages

handle_info is invoked whenever an Elixir message reaches the channel.

Session-based authentication makes sense for request/response–type applications, but for channels, token authentication works better because the connection is a long-duration connection. With token authentication, each user gets a unique token. Tokens allow for a secure authentication mechanism that doesn’t reply on any specific transport. Once the user is already authenticated using the request/response traditional approach, the application can expose the token to the frontend, and this token can be used by the channel.

Ch. 11: OTP

OTP is a way to think about concurrency and distribution, using patterns that allow developer to use concurrency to build state without language features that rely on mutability.

OTP is the name of the library encapsulating concurrent state and behavior, and the abstractions is called generic server, or GenServer.

Supervisors need to be able to restart processes in the right way, according to policies that are best for the application. For example, suppose a database was upgraded, and the connection stopped. The application would need to automatically kill and restart the connection pool. This policy should not impact the code that uses the database. If developer can replace a simple supervisor process with a supervisor tree, we can build much more robust fault-tolerance and recovery software.

In Phoenix, there is little code attempting to deal with every possible error exception. Instead, we trust the error reporting to log the errors so developer can fix what’s broken, and in the meantime, supervisor tree can automatically restart services in a last good state. OTP captures these clean abstractions in a coherent library, allowing developer to declare the supervision properties that most interest us. This allows developer to build robust self-healing software without building complex self-healing software.

A supervision strategy is what policy is used when a process dies, for example, the :one_for_one policy will restart a child process when the child dies. If all resources depend on a common service, developer could specify :one_for_all to kill and restart all child processes if any child dies. Developer doesn’t need to add any code to fully supervise every process, only configure a policy to tell OTP how to handle each crash.

The default restart strategy is :permanent, the supervisor always restarts child processes with this strategy. With :temporary, the child process is never restarted. Under :transient, the child is restarted of if it terminates abnormally, with an exit reason other than :normal, :shutdown, or {:shutdown, term}.

:temporary is useful when a restart is unlikely to resolve a worker crash, or when restarting doesn’t make sense based on the flow of the application.

Child processes can have different restart strategies, supervisors have configurable supervision strategies.

:one_for_one if a child terminates, the supervisor restarts only that process
:one_for_all if a child terminates, a supervisor terminates all remaining children, the restarts all children
:rest_for_one if a child terminates, a supervisor terminates all children defined after the one that died, then restarts all terminated processes
:simple_on_for_one Similar to :one_for_one but used when a supervisor needs to dynamically supervise processes. For example, a web server would use it to supervise web requests, which may be 10, 10,000, or 100,000 processes.

A channel in Phoenix is an OTP process built to serve a single user in the context of a single conversation on a topic.

When fetching real-time (pp 212) data about a video that will be synched in real-time, a network or failure by the 3rd party can occur, but since we’re making multiple requests, we can ignore the failure and use the responses from the successful requests. The restart strategy spawns multiple processes and shouldn’t restart the failures because its time sensitive, the errors can simply be moved past, we’ll accept any successful requests that return, so we’ll use the :temporary restart strategy.

OTP applications protect in both directions, that is, if the Phoenix server itself crashes, we bring down the children processes and restart them so no resources are leaked. When a child process crashes, we bring down that process and restart it so we can attempt clean recovery. When building a supervisor and children, the process of defining the restart strategy and supervision strategy lets developer focus on the main application logic, as any unknowable error handling is already taken care of. Let the errors crash gracefully, and OTP will handle the rest.

Because GenServer’s are meant to be generic servers, they hold both computation and state. However, in many cases, we want a process to store state only, or only execute a particular function. An agent is a simple GenServer

Chapter 12: Observer and Umbrellas

Umbrella projects allow developer to develop and test multiple child applications in isolation side by side while providing conveniences for configuring and building them only once. Instead of breaking the application into multiple applications in distinct source-code repositories, which adds too much overhead to the development workflow, you can use Umbrella projects. The combination of Mix and OTP make this abstraction a good one for separating core concerns and dealing with complexity.

Observer is an Erlang tool for understanding all running processes in an application. To start:

iex -S mix
> :observer.start

The tool visualizes different aspects of the application, lets developer see a list of all running processes, how much memory, messages the system is using. Developer can more easily see where bottlenecks occur by finding processes with large numbers of messages.

Each Umbrella has a parent directory that defines

The shared configuration of the project
The dependencies for that project
the apps directory with child applications

to create new Umbrella projects, use mix new <name> --umbrella outside of your application, which stubs out a simpler Phoenix app, this project can then be moved inside the parent project.

Chapter 13: Testing Channels and OTP

Stubs and mocks are both testing fixtures that replace real world implementations. A stub replaces real-world libraries with simpler, predictable behavior. With a stub, developer can bypass code that would be difficult to test. Stubs should have nothing to say whether a test should pass or fail. Stubs are a simple scaffold implementation standing in for a more complex real-world implementation.

A mock is similar, but replaces real-world behavior just as a stub does, but it allows developer to specify expectations and results, playing back those expectations at runtime. A mock will fail a test if the test code doesn’t receive the expected function calls. A mock is an implementation that records expected behavior at definition time and plays it back at runtime, expecting those results.

Within the Elixir community, avoid mocking whenever possible. Most mocking libraries end up changing global behavior—for example, by replacing a function in the HTTP client library to return a particular result. These function replacements are global, so a change in one place would change all code running at the same time. That means tests written in this way can no longer run concurrently. These strategies will snowball, requiring more and more mocking until the dependencies among components are completely hidden.

A better strategy is to identify code that’s difficult to test live, and to build a configurable, replaceable testing implementation rather than a dynamic mock. The development and production code will use the simple :httpc client, and the testing code will use a stub that is called during the tests.

A major advantage of writing asynchronous tests in OTP is the tests run concurrently, meaning they can be run in parallel, so the entire suite of tests can finish more quickly than their synchronous counterparts.

References

McCord, Chris. Tate, Bruce. Valim, José. (2016). Programming Phoenix. The Pragmatic Programmers. Print.

Since the advent of the first banner-ads on the web, advertising has grown to become the most popular business model, and with the growth of advertising there has been an equal expansion of tracking techniques. These techniques include tracking users clicks, the duration of site visits, metrics about device capabilities, the variety of sites visited, and the frequency of these visits are only the bare-minimum for todaysʼ tracking analytics. What is considered “business as usual” today includes a plethora of intrusive techniques that were previously considered to be spyware. This includes techniques such as mouse movement recording, keylogging, geolocation tracking, and even video recording of what the user does on every web site they visit.

Current tracking practices are intrinsically based on a disrespect of peoplesʼ personal and private data. Tracking companies business models are based on the presumption that 1. You forfeits your right to privacy in exchange for ad-subsidized content, and 2. Your personal data belongs to them. These conditions are not sustainable in the long-term if we wish to maintain the right to privacy in the future. Privacy will continue to be erroded for all classes of society, but will especially impact the poor, who cannot afford to opt-out of ads in exchange for a subscription.

Overview of Web Advertising

Tracking practices arose from the last twenty five years of the web, during a time when the vast majority of web content was subsidized by banner ad-based business models. These models naively depended on the collection of increasingly intrusive metrics in order to precisely target consumers. As ad markets became more efficient over time, metrics needed to become more precise in order to keep advertising conversion results positive. Itʼs an arms race between the ad-networks competing to deliver ads with better and better conversion rates. This model of increased tracking and surveillance in exchange for optimized return on investment has been called “Surveillance Capitalism”.

Ad-networks are middlemen who facilitate a “triangle trade” between content-providers, advertisers, and users. In this system, a content provider (e.g. The Wall Street Journal) wishes to publish a news story to their readers. Their costs include paying reporters, hosting the website, and they wish to be paid by an advertiser. The advertiser (e.g. Chase Bank) wishes to get attention to their product (e.g. a new savings account), by placing an ad in front of users likely to open a savings account. The advertisers costs include designing an ad, and paying the ad-network to place the ad on sites within the ad-network (i.e. ad-networks have many sites, one of which is wsj.com). The user wishes to read the latest news (preferably for free). The ad-network is a match-making company, they (via tracking analytics) are able to see which users visit the sites in their network of content-providers, and are able to instantly place an ad for the Chase savings account to the user of the Wall Street Journal who has a metrics profile most similar to other users who have opened a new savings account recently. The ad-network matches advertisers to the users most likely to buy something from the advertiser. When the ad is placed, the user sees the ad, the ad-network is able to inform both Chase bank and the Wall Street Journal that the ad was seen. Finally, the ad-network is paid by Chase, and the Wall Street Journal is paid by the ad-network for hosting the ad.

Users “pay” for the websites they visit in hidden ways. They pay by implicitly allowing ad-networks to build a marketing profile about themselves. Their browser automatically sends information to the ad-networks about their browsing behaviors. This happens without users consent or knowledge. This data is shared between the ad-networks, then aggregated and combined to create detailed profiles containing lists of sites users have visited over time, including what theyʼve done on those sites. This profile is then used to efficiently advertise products and services to the user.

Users also pay for the amount of data ads consume on their smartphone data plans. For example, on TMZ.com¹ there are 47 different trackers, 42 different cookies, and a single page request to the homepage takes 10.02 megabytes of data across 262 requests. This request averages 52 seconds to load. The user-experience is slow, and the homepage doesn’t allow the user to read a single article. With an ad-blocker installed, the site payload was only 3.87 megabytes of data across 104 HTTP requests, averaging 9.76 seconds to load. This is a significant speed and data improvement, and showcases the reduced data savings. Up to 50% of a users mobile-phone data plan is used for serving ads, each ad taking ~5 seconds to load, and decreasing overall phone battery life by ~20%. The top 1-million sites use an average of 25-30 trackers.

Users “pay” for content with their security as well. In 2016 malware being served via ad-networks was up 132%².

The problem with current tracking practices is that users are not empowered to make consensual decisions around how their online behavior is collected and aggregated. Users are unaware of the sacrifices they’re making to view content online.

Do Not Track (DNT)

Do Not Track is a technology and legal framework that enables users to opt out of tracking by ad-networks, analytics services, and social platforms. DNT empowers users with a choice they currently don’t have. It’s a feature in web browsers that allows users to express their preference for not being tracked to the web sites and services they use every day. A new set of laws including the EU GDPR and California AB 370 provide a legal regime to enforce the respect of this new browser preference. Combined, the technology and law provide a viable path forward to reclaim the right to privacy on the web.

DNT is a user making an explicit feature request, I do not want to be tracked. It is a better version of the Do Not Call registry. DNT is a user preference that forces the browser to send an HTTP request to the server explicitly telling that server not to track user behaviors.

Privacy is a Feature

People (generally) don’t care about privacy until they’re affected by its’ loss. A few examples; people provide their email addresses to get a receipt at a brick-and-mortar store after a transaction, but hate when they receive spam. People use credit cards on sketchy websites, but hate when they see fraudulent charges. People reuse the same weak passwords on multiple websites, but hate when they get hacked. The problem is an expectation of privacy when they’ve taken actions that undermine their privacy.

DNT and the laws that enforce respect for DNT enable users to take action to express their right to privacy. Web companies and services that respect it have an opportunity to benefit. Companies like Apple are setting a good example of this mindset towards privacy by building pro-privacy features and services.

Privacy marketing will be an important value-add for Internet commerce. It will be a terrific way to gain market share at the expense of the competition — or to lose much of your market share, if you find yourself on the wrong end of a privacy campaign

Organizations that respect and offer a straightforward choice around tracking to their customers will earn their trust and loyalty.

“great experiences don’t have to come at the expense of your privacy and security. Instead, they can support them.”

Apple seems to have wised-up to selling privacy as a component of a luxury product. They have marketed features like TouchID, ApplePay, encryption and differential privacy as privacy-securing features. At the same time they have undermined privacy in numerous ways.

The EU GDPR and AB 370

Browsers and servers today can implement technological support for DNT, but until recently, ad-networks had no incentives to honor DNT. The ad-networks continue to ignore the DNT request, and continue tracking users either explicitly or implicitly. They had clear disincentives to ignore the user-preference.

California law AB 370 makes it more difficult to continue to ignore DNT without explicitly excluding Californian residents. The law, (regardless of where the website is hosted) modifies the California Online Privacy Act. Under SECTION 1, Section 22575 requires a site to disclose how it responds to the Do Not Track preference.

SECTION 1. Section 22575

(5) Disclose how the operator responds to Web browser “do not track” signals or other mechanisms that provide consumers the ability to exercise choice regarding the collection of personally identifiable information about an individual consumer’s online activities over time and across third-party Web sites or online services, if the operator engages in that collection.

(6) Disclose whether other parties may collect personally identifiable information about an individual consumer’s online activities over time and across different Web sites when a consumer uses the operator’s Web site or service

(7) An operator may satisfy the requirement of paragraph (5) by providing a clear and conspicuous hyperlink in the operator’s privacy policy to an online location containing a description, including the effects, of any program or protocol the operator follows that offers the consumer that choice.

This means sites need to explicitly say whether they honor or do not honor the user expression of DNT. This doesn’t go far enough to incentivize companies to actually honor DNT, it merely incentivizes them to disclose how they address the preference. Ultimately this will not empower users in any practical way.

EU General Data Protection Regulation (GDPR) was designed to harmonize data privacy laws across Europe, to protect and empower all EU citizens data privacy and to reshape the way organizations across the region approach data privacy. It was approved by the EU Parliament on April 14, 2016 and entered in force on May 25, 2018 - at which time those organizations in non-compliance could face heavy fines.

You may have already seen the preemptive effects of the EU GDPR in the wild, such as the “cookie disclosure” notice on many sites in the EU and UK: “Yes, I accept cookies”.

The EU GDPR law goes much farther in that it gives users actual choice rather than just disclosure. To summarize the law, it applies to all companies processing the personal data of data subjects residing in the EU, regardless of the company’s location. It also applies to a controller or processor not established in the EU, where the activities relate to: offering goods or services or monitoring of behavior of EU citizens. Non-EU businesses processing the data of EU citizens will also have to appoint a representative in the EU.

Organizations in breach of GDPR can be fined up to 4% of annual global turnover or €20 Million (whichever is greater) of the previous year. Most serious infringements are not having sufficient customer consent to process data, or violating the core of Privacy by Design concepts. Companies will no longer be able to use legalese terms and conditions, that is to say, request for consent must be in plain language. Any data collection and processing must be directly attached to the point where consent is given. Companies can’t just collect information for any purpose, the purpose must be clearly limited and stated. Finally it must be as easy for consumers to withdraw consent as it is to give it.

The GDPR includes the “Right to be Forgotten”, also known as “Data Erasure”. It entitles the user to have the data controller erase personal data upon demand as well as cease further dissemination of the data. It also forces third parties to halt processing of the data.

“Data Portability” is the right for a user to receive the personal data concerning them, which they have previously provided. It must be in a “commonly used and machine readable format”. The user has the right to transmit that data to another controller of their choosing.

A major component of the GDPR is companies can no longer “Collect it all now, use it later”. Under Recital 39, it notes that “specific purposes for which personal data are processed should be explicit and legitimate and determined at the time of the collection…” Recital 32 notes that “When the processing has multiple purposes, consent should be given for all of them…” If the data is later to be used outside the scope of original purpose, user must receive an explicit opt-out opportunity accoding to GDPR, Article 6, paragraph 4, Recital 50. The language of the original purpose must be in clear, plain language (not legalese) according to GDPR, Article 13, paragraph 1, c.

These regulations together provide a much more user-centric set of rights that were previously impossible for users to exert over data collectors. Together with the DNT feature, users can finally push back against online tracking.

Technology Pushes Back Against Tracking

DNT and anti-tracking technologies are making the previously hidden practices of web tracking visible to the end-users. This technology will (hopefully) help shine light on the seedy underbelly of intrusive advertising. It’ll make the invisible visible. These changes will empower users to make informed decisions that were either technically difficult, or outright hidden.

Safari web browser has a new feature called “Intelligent Tracking Protection” or ITP, which purges third-party cookies that are determined to be able to track users across sites. After 24 hours, the third-party cookies can only be used for log-in purposes if the user visits that site as a top-level domain. All cookies are purged after 30 days. The software utilizes a machine learning model to classify which top privately-controlled domains have the ability to track the user cross-site, and those determined to be trackers will be deleted from the browser.

Firefox will soon allow users to prevent tracking cookies in the “Tracking Protection” feature. Eventually, cookies set without explicit DNT consent will be blocked if the user has enabled Tracking Protection. It remains to be seen if this feature will be enabled by default. Firefox 57 introduced a change to how the browser loads scripts matching domains in their Tracking Protection database. This has the effect of reducing the perceptual page load time for the 1st-party site, and making tracking ads load more slowly.

Google Chrome will begin filtering obtrusive ads by default in early 2018.

Chrome filtering will work by the Google search engine conducting an “ad experience review”. Reviews are periodic, and result in a “passing”, “warning” or “failing” grade for the site under review. The ad review engine crawls a subset of site pages, conducts “user-like” interactions, and compares ad experiences to their “better ads standards”. If the site detects violations of the standard, the report will result in a “failure”. Sites with ads with a failing Ad Experience Report will have their ads filtered in Chrome. It remains to be seen if this feature will be enabled by default.

Chrome ad filtering is controversial because Google has a direct conflict of interest in that its ad filtering will punish the ad-networks that are in direct competition to the Google-owned Doubleclick ad-network. It is in Googles’ best interests to filter ads from their competition, such that it incentivizes advertisers to use the Doubleclick network since Chrome would be filtering ads outside the Doubleclick network. This could result in monopolistic behavior on the part of Google.

A new player in the browser market is the Brave Browser. Brave is founded by Mozilla founder and creator of JavaScript Brendan Eich. Brave provides an “ad-less” web experience. It enables a direct monetary relationship between the content-provider, advertisers, and their user audience (replacing existing ad-networks). Brave browser will include a new cryptocurrency called the Basic Attention Token (BAT) that facilitates a marketplace directly connecting content-providers, advertisers and users. Users can opt to directly pay the content-provider for content using the BAT token. Alternatively, users can be paid BAT tokens in exchange for seeing ads placed by the advertisers participating in the Brave ad-network. These BAT tokens can be used to pay for content created by content-providers. Brave has been tight-lipped about how exactly users privacy will be better protected on their ad-network compared to existing ad-networks.

Another recent entrant into the browser space is Better, a browser built according to Ethical Design concepts. This browser eschews “better ads” standards that companies like Google, Brave, Ghostery, and AdBlock Plus are promoting. Instead, it offers a product thats’ interests are aligned directly with consumer interests because they are paid directly by its’ users, and the business model doesnʼt rely on surveillance capitalism in order to make money.

Preparing for the Future

There is potential for the changes described above to have wide impacts on the advertising industry, and I have some recommendations for organizations that wish to get ahead of these changes in order to reduce risk and lower the force of the impact.

• Design with privacy as a guiding principal. It’s far easier to handle privacy considerations in the design phase rather than having to backtrack a product once built.
  • Design humane technology that respects users privacy.
  • Consider where users privacy is placed at risk.
  • Design features that achieve a goal with a smaller or no privacy impact.
  • Design data collection features that ask for informed, explicit consent.
• Differentiate your organization by highlighting a privacy-respecting approach to data collection.
• Be a good corporate citizen by empowering your users with choice around how their data is collected.
• Allow users to both export their data, and delete their accounts from the product.
• Honor the DNT browser setting according to the EFF DNT Guide.
• Describe in plain language how users data is collected and used.
• Reduce the amount of metrics stored internally to the minimum possible, and purging that data whenever possible.

With technology and law moving forward in lockstep, I see a positive path forward for the protection of the right to privacy online.

system-font-i18n-css is optimized to select the best system font on a per-character basis, based on the unicode range of that character.

system-font-i18n-css provides twelve variations for the Sans-serif family of system font. This font stack provides more consistent character typesetting across multiple languages and all modern operating systems.

system-font-i18n-css is optimized to select the best system font on a per-character basis, based on the unicode range of that character.

This project aims to fix that. Docs. Available on NPM as system-font-i18n-css and on github as system-font-i18n-css

I included React, Angular and Ember to the list of contenders for comparison. We read the docs, implemented ToDo MVC apps, prototypes, and reviewed the community. We produced comparison spreadsheets and went to local meetups. We reviewed design docs and discussed the merits of each approach. Finally, we discussed our findings. Both Ember’s FastBoot and React’s server-side pipeline were not yet ready for production. The Angular community was in flames from the v2 rewrite. Angular 4 was yet another rewrite that was shaping up to look disastrous. The core Angular team had abandoned ship. Instead of making a final decision on a framework, the team wanted to wait until we understood more of our applications use cases.

TL;DR: We decided to defer the decision. Instead we wrote the app with a “NOJS-first” approach. The implementation was simple, using HTML forms. Using progressive enhancement, we could add a richer frontend in the future.

The experience investigating Ember stuck with me. Ember was using web-standards and best practices. Convention-over-configuration came from Ruby on Rails. It borrowed performance UI rendering strategies from React. It had a first-class testing and documentation story much like Django. Ember was distilling the best ideas and practices and placing them at the core of their project.

A year later, our decision was working out well, and the project was humming along. It was well-tested, performant, and had frequent new feature additions. I decided a change of pace would be fun and challenging. The app was in good hands.

When starting my job search, I focused on companies that were using Ember. One of the major considerations for my next job was getting my hands deep into Ember. In November 2016, I joined a startup using Ember to build a personal finance application.

Almost a year of working in Ember, I’d like to summarize my thoughts about what makes it a great framework. There are a million articles comparing the big JS frameworks; this isn’t one of those. This article focuses on what keeps me happy about the decision to continue using Ember.

Ember feels familiar to anyone coming from developing basic HTML/JS/CSS sites, WordPress, Ruby on Rails apps, or even Django apps. You have template files, you have JavaScript files, and you have CSS. You’re not writing markup, JavaScript or CSS in JS files. The templates are much like Mustache. It feels like you’re writing code for the web.

You spend little time in configuration relative to building application features. The framework defaults work well. Ember’s architecture is better known as “convention-over-configuration”. The core team dedicates time and resources to understand how the community uses the framework. This helps them bake-in best practices as the defaults.

There is a single “Ember way” of doing things with the framework. This aspect of Ember is one of its biggest benefits. When the community grapples with problems that span applications, the core team distills common solutions by defining well-crafted abstractions. It’s amazing how quickly an opinionated framework can help you see patterns in your project, and solve problems using framework abstractions. These abstractions also make it easier to follow best practices without thinking about it. Good examples of these are the build system, the pluggable deployments system, and routing. In the end, developers focus on building features rather than making many small decisions around 3rd party libraries and app configuration.

Ember has a bigger learning curve than React, but smaller than Angular. This statement is misleading because Ember is solving more problems than React. React is only the ‘V’ in MVC, while Ember is solving the problem of building complete applications. A fair comparison would be between the learning curve of Ember vs React with Redux, Apollo, React Router, Koa, Relay, and Webpack. Ember provides a solution as well as a converging ecosystem for building modern web applications. Ember’s design is to solve application-level problems. This means no liabilities around custom code that wires libraries together. This design eliminates a class of problems where library bounderies don’t connect. Worse even, is when inconsistencies get introduced because libraries have overlapping concerns.

Web standards are important in the Ember community. The core team adds new features with an eye towards standards organizations such as the W3C and ECMAScript TC39. Features are first adopted in an Ember-specific way (i.e. Ember.RSVP); allowing developers to use these features before the standard has landed. Eventually these “welcome mats” get replaced with the final standards-based implementations. This ensures that the framework will not only keep pace with, but stay ahead of and influence the standardization process. This also adds a degree of future-proofing into the framework.

Ember makes it easier to stay up to date with the latest and greatest framework features. With an all-in-one architecture, everything is well tested, and works together as versions move ahead. Addon authors can use packages like ember-try to test against various versions. Finally, the Ember team takes a responsible approach as it moves along the roadmap (it has a roadmap!). The project has a long-term vision, and the core libraries align their roadmaps on a 6-week release cycle. The release team flags deprecated features and wrapping new ones in feature flags. This allows developers to safely test out new APIs and responsibly sunset deprecated ones.

CSS organization becomes trivial if you let the CSS follow Ember routes and components. Pair any page-oriented CSS with the route-template, and pair the majority of your CSS with components. Combine this with a solid CSS framework like Bootstrap, Foundation, Bulma- and you’ve made long-term CSS maintenance easier.

After a year of working in Ember, attending EmberConf 2017, visiting meetup groups, and talking with folks in Slack, I’m sold. It’s a pleasure to work in, and the roadmap ahead points to a bright future. Tree-shaking and improvements in static analysis are shaping up to help break the app into smaller pieces for faster delivery over the wire. I look forward to the next year using Ember.

Available on NPM at ember-i18n-changeset-validations and on github at ember-i18n-changeset-validations.

Before Trump takes office, I want to create a picture of the high-water mark, something I can point to and say “That was how it was” without the fog of time, or the distortions of propaganda and ideology. I was inspired by Sarah Kendzior:

Fellow Americans, I have a favor to ask you. Today is November 18, 2016. I want you to write about who you are, what you have experienced, and what you have endured. Write down what you value; what standards you hold for yourself and for others. Write about your dreams for the future and your hopes for your children. Write about the struggle of your ancestors and how the hardship they overcame shaped the person you are today. Write your biography, write down your memories. Because if you do not do it now, you may forget. Write a list of things you would never do. Because it is possible that in the next year, you will do them. Write a list of things you would never believe. Because it is possible that in the next year, you will either believe them or be forced to say you believe them.¹

The Spectacle of 2016

The 2016 election was a farce and a tipping point where politics have finally become an extension of what Guy Debord called “The Spectacle”². Both “viable” candidates offered the same neoliberal flavor of poison. It was a false choice between “Coke vs Pepsi.”

Clinton campaigned on a platform powered by a misplaced faith³ in Big Data⁴. She spoke in a language of clinical rhetoric to nobody except Wall Street. She was a status quo candidate that lacked vision, offering nothing except a hedge against the fascism of Trump. Her speeches were empty, dispassionate and simply disengaged from reality. Her campaign and the DNC were corrupt from the start, and they didn’t try to hide it.

In my view, Trump is an angsty teenager who uses knee-jerk reactions and emotion to speak to those disgusted, disenfranchised, and ignored by the Washington Consensus. The Republican party nearly destroyed itself by failing to field a status quo candidate from their own ranks. This opened a gap which Trump was able to fill by speaking directly to the middle class fear of sliding into the lower classes. This fear has been stoked by mainstream media after the economic crisis that started in 2008 and has continued up to today.

The candidates lie, and we know they’re lying, and they know we know they’re lying. It doesn’t matter because there is no longer an alternative vision for the future and there is no longer any accountability for those in power.

If one thing is clear from this election, it is the contradictions inherent in the policies that Trump wishes to enact, as well as the contradictions inherent in Neoliberal Capitalism:

• Trump believes that government regulation is preventing economic growth in the US and wishes to deregulate. Yet regulation is the people’s best means to prevent corporate abuses of power: It can be used to prevent job flight abroad, as well as prevent tax-shelters, and incentivize hiring and manufacturing at home.
• Trump says he is against war abroad, and yet wants to deploy more troops to Iraq and Afghanistan.
• An inability to account for economic contradictions such as “overproduction relative to demand”, with unemployment due to automation and a shrinking of discretionary spending in the middle class.
• Military contradictions including rising tensions between the US, NATO, and its military allies, as shown with the recent spying scandals revealed by Edward Snowden; along with the increasing economic unsustainability of war.
• Cultural contradictions including tensions between U.S.A. Judeo-Christianity, Islam, and other minorities as we see a rising trend of anti-Muslim sentiment growing in the “Alt-Right”.
• Social contradictions encompassing the increasing gulf between the so-called ‘American Dream’, the belief that everyone can prosper in America through hard work, and the reality of American life (the fact that more and more people can’t).

It seems Trump could actually be a less “hawkish” president than Clinton (who just wants to go to more wars to fill the coffers of companies involved with the military industrial complex). Trump instead seems to want to “do business” with Russia and China. The fear I have for Trump is not the “Hawkish Empire” that Hillary Clinton would’ve brought with her, but instead a form of “Corporate Fascism” where he pours fuel on the fire of trans-national capitalism, while funding the NSA/FBI/CIA to go after those who dissent against a corporate-sponsored deep-state.

In such a Intellio fascist nightmare, the U.S.A. uses it military might to act as the private security arm of Boeing, Microsoft, Google, Facebook, Chase & JP Morgan. We will see a deeper criminalization of copyright infringement, boycotting, and more forced-purchase agreements like the current “Mandatory Health Insurance” programs that force taxpayers to use their income to buy big-corporate services. At the same time, we will see a gutting of “commonwealth” programs like the ACA, and Planned Parenthood. These programs will not be abolished, but will instead be privatized much in the same way vast portions of the Russian national state was privatized after 1989. In its wake we will see the rise of a new corporate oligarchy.

This election I made a conscious decision not to debate my coworkers, friends and family over the candidates or “the issues”. The 2016 campaign pitted two candidates against each other in a way that drastically simplified and framed discourse in moral absolutes. These absolutes were curated and made digestible for mainstream media and social media.

Only the ignorant deal in moal absolutes. Debate isn’t debate when “the issues” are carefully curated to frame the dialog as a decision between choosing evils when they’re both evil. Taking sides can only end in animosity towards each other, and creates wedges in the very communities we need to be building for when the time comes to fight against corporate-fascism. Most importantly, we lose sight of the bigger picture when the majority is split across artificially imposed lines. Now is the time we should be gathering forces in our local communities with the friends, coworkers and families that actually impact our daily lives. Think global; act local is a cliche, but it works, and it’s our best defense against the shitstorm to come.

At Present

I can freely drive from home to any state in the US, without interference from the police or border patrol. Border patrol thinks they can have their way with folks along the border, but it’s just a hassle. TSA wastes time with security theater at airports, but I can travel to any country without problem. Be polite, say as little as possible, and the headache goes away.

When planes fly overhead, I sometimes look to the sky and wonder where those travellers are going. I don’t worry about drone strikes at home. When I hear police sirens, I typically don’t even notice, its probably a traffic infraction. Don’t talk to the police, but be polite and the headache (usually) goes away.

I don’t believe that immigrants are rapists, killers and drug dealers. I believe immigrants today are no different than the immigrants that built this country; they’re just people trying to take the opportunity to improve their lives.

I don’t believe that terrorists want to fight us because they “hate our freedoms” but instead they want to fight to protect their own way of life. I believe terrorism is caused by a reaction to U.S. policies in the middle east funding the (primarily) Saudi Arabian royal family. The royal family receives military equipment to oppress its own people and extract local natural resources (oil) at the expense of its own people. The royal family also sends money to the Salafi movement, an ultra-conservative regressive form of Islam that is destabilizing the middle east and is the root of Al-Queda, ISIL and the current war in Syria. In exchange for a market for Saudi oil and military supplies, the U.S. indirectly empowers the wider Salafi movement to fight against the West while companies like Boeing, Lockheed Martin, Pratt & Whitney, KBR, General Electric, Honeywell, Raytheon, General Dynamics, Northrop Grumann profit from demand generated by continual wars.

I refuse to succumb to divide and conquer tactics; when the oppressor uses arbitrary difference to separate humanity and then pit them against one another by granting one side rights and denying those rights to the other. These are apartheid tactics. Separation, isolation, and alienation are inhumane tactics that serve the interests of authority and power. The differences used to separate can be based on race, nationality, religion, gender, sexuality, class, political party, and merit. These differences should be embraced and celebrated, not used to divide us. We are all 99% similar, and the differences are what make us unique. If they attack our minorities, they attack us all. Our strength lies in unity against bigotry, hatred, and fear. We are united as one people for the good of all.

I refuse to engage in consumerist activism. I wont be buying Che Guevara t-shirts to lend my personal brand more progressive looking street-cred. I wont be cheering for corporations making donations to political causes when it serves their bottom line⁵. These types of activism are counterproductive as they subjugate real social activism via personal relationships into a relationship mediated by physical goods and associations to brands.

I refuse to acknowledge the DHS, CBP as necessary or helpful to US national security, as defenders of the constitution or more generally as upkeepers of the rule of law. At their worst, they’re oppressive thugs for a racist deep state.

I will express my opinions and disagreements with the U.S.A government as I always have. I will continue to be anti-war, anti-drone, anti-neoliberal, anti-imperialist, anti-capitalist, and violently anti-fascist (if the fascists seek to impose oppressive rule here in the U.S.A.) Censorship isn’t political in the U.S.A. We address censorship as a market opportunity. When folks are pirating movies, companies address that gap by providing a (relatively) cheap service to make it convenient to pay for the movie. The rhetoric of alarm around “Fake News” is alarming however. It’s beginning to sound eerily similar to the rationale the Chinese government is using to censor social media, the “danger of rumors” causing fractures in social cohesion. I’m worried the Trump administration will take a similar path.

I will oppose the reappropriation of language. I will not use terms they redefine, nor use E.g. ‘fake news’new terms they make. We must preserve the true meaning of words, and reconsider the use of language to simplify discourse and bias our perspectives.

The media, press, bloggers and social media are free to criticize the political establishment; it’s a national pastime. The Daily Show and Colbert Report are built on getting laughs from the deep cynicism of the Left. I post my criticism to my site, twitter, etc. and don’t expect the police to visit my house, my work, or to worry about them asking about the post the next time I get pulled over for a traffic infringement or at border control. I’m relaxed when expressing my views, I go to protests and engage in local politics. I consider these acts to be part of my civic duty and expect the government to NOT ask about my activities. An expectation otherwise would result in the erosion of civic activity due to self-censorship- the most pernicious form of censorship because the censorship has become internalized.

My associations with friends, family, coworkers and neighbors are private, and I expect them to remain private. I worry about my peers, who continue to divulge the nature of their personal relationships and love-lives to social media companies. This is the slow commodification of relationships. I worry that these companies (Facebook, Tinder, etc) are increasingly mediating cherished relationships between people.

Obama deported more undocumented migrants than any previous administration, he unilaterally dropped ~26,000 bombs on Syria, Libya, Afghanistan, Yemen, Iraq and other countries. I expect this to continue. I don’t worry about going to war, either at home or abroad. War has been a venture for the Washington Consensus to control foreign access and control to natural resources and access to markets for our trans-national corporate interests. This impacts the poor, those who volunteer to fight, and immigrants who see a path to citizenship. The Consensus seems wise enough to know the majority is war-fatigued, and fear protest or resistance if a draft was ever reinstated. I don’t expect the middle class to rally behind Trumps wars, and I certainly will never support his wars.

The worst aspect of the Obama administration is that it quelled the energy of left-oriented grassroots movements. People thought “Oh, Obama is like me, hes smart and he’s got this covered.” It kept the Left at ease, and this mindset resulting in eight years of political inaction.

The one redeeming side-effect of the Trump administration is that it will reveal the true nature of modern neoliberal power politics in the United States. Trumps’ administration will brazenly take action without the pretenses of conciliatory compromise, or mutual respect for any existing political party or an adversarial press. His administration will be the “Big Reveal” of the level of corruption in our body politic. Trump will accomplish this by placing the big influencers (who, up to this point, have been outside advisors) in direct positions of power; e.g. Instead of having Exxon Mobile acting as an external industry advisor, Trump will make him the CEO the Secretary of State. These acts will tear down the facade of a disinterested political apparatus and reveal the true nature of politics rigged from top down to reward the biggest corporations. This will help the Left appeal to the middle by pointing out readily apparent and brazen corrpution.

During difficult times, there is a blossoming of arts and culture in response to authoritarian oppression. While I have no expectations for positive reform in federal politics, I have high hopes for local community action and the arts.

Massaging the SVG File Format

First things first, I took the source image and ran it through the delaunay triangulation - image triangulation experiment software, experimenting with various values for blur, accuracy and point count. The output of this software was a 512Kb SVG file.

Next I opened the file in Inkscape, a SVG vector graphics editor to remove the polygons in the background, cleanup some awkward polys and make a few minor adjustments. Inkscape adds a bunch of metadata and additional properties to the raw SVG that can be cleaned up.

Once I was happy with the results of my tweaking in Inkscape, I saved the SVG and opened it in SVGOMG a web-based SVG optimization GUI for the SVGO suite of tools. This tool will output a much smaller SVG file that is ready for either editing or delivery over HTTP. In this case, the file was reduced to 312Kb a 40% reduction in file size, primarily by transforming point-based <polygon> elements to the more concise to <path> element.

Animation

With the SVG file ready, I moved on to the action animation work.

The animation would be a series of CSS keyframes that would simply toggle the opacity level to produce a flickering affect. The artwork begins with all opacity set to 0 (transparent), and each keyframe incrementally toggles opacity until the last frame finally sets it to 1.

.p {
  /* each polygon "path" starts fully transparent */
  opacity: 0;
}

/* a set of keyframes that incrementally toggle opacity, this */
/* keyframes set is called "slide-into-place" */
@keyframes slide-into-place {
  0% {
    opacity: 1;
  }
  5% {
    opacity: 0;
  }
  15% {
    opacity: 1;
  }
  17% {
    opacity: 0;
  }
  50% {
    opacity: 1;
  }
  55% {
    opacity: 0;
  }
  60% {
    opacity: 1;
  }
  75% {
    opacity: 0.25;
  }
  85% {
    opacity: 0.85;
  }
  90% {
    opacity: 0;
  }
  95% {
    opacity: 0.25;
  }
  100% {
    opacity: 1;
  }
}

Every <path> in the SVG document is given a classname p (for path), I then use javascript to get references to these paths, and simply add a new classname is-animated to their class attribute. This classname will be used to attach the keyframes to the SVG paths.

.p.is-animated {
  /* sets the keyframes to use for animation */
  animation-name: slide-into-place;
  /* pause before animation starts */
  animation-delay: 1s;
  animation-timing-function: ease;
  animation-direction: normal;
  /* Run the animation only once */
  animation-iteration-count: 1;
  animation-fill-mode: forwards;
}

To attach the keyframe animation to the SVG <path>, the only think necessary is to add the is-animated classname to the element. We do this on Line-A. This is done inside a loop that iterates over all the <path> elements.

If I just add the is-animated class to the element, all polygons will flicker at the same time, looking like the entire image is flickering- which isn’t what I want. To address this, I need to randomize the start-time of when each polygon begins flickering. This makes each polygon flicker on its own timeline. I get a random number on Line-B and use that as a delay to begin the flickering.

path.style.animationDelay (on Line-C) is used to wait for a randomized timeout (Line-B) period before animation starts.

function init() {
  var paths = document.querySelectorAll("path"),
    i = 0,
    randTimeout = null;
  path = null;
  for (i = 0; i < paths.length; i++) {
    let path = paths[i];
    randTimeout = Math.floor(Math.random() * (1500 - 1) + 1); // Line-B
    path.style.animationDelay = randTimeout + "ms"; // Line-C
    path.classList.add("is-animated"); // Line-A
  }
}

The above init() function is called once the SVG document has finished loading.

<svg
  xmlns="http://www.w3.org/2000/svg"
  viewbox="0 0 660 660"
  onload="init()"
></svg>

Result

(Reload the page to view the animation again)

In 2011 I wrote:

Travel opens my mind to new perspectives and allows me to recontextualize existing systems of thought.

While true for me at that time, for that trip; there is nothing magical about travel itself that opens the mind to new perspectives nor helps to “recontextualize existing systems of thought”. One must be ready and “in the mood” to open ones mind and experience life from a new perspective.

The distance travelled is of no importance when the most distant places on Earth can be connected to within a 24-hour period and a few short layovers. The travel doesn’t even begin until you have arrived at your destination. Perhaps it’s how the distances are percieved that really matters. If ones perception of 2,400 miles is two mediocre movies and a few bad in-flight meals, perhaps it’s not really “travel”.

The air traveller only travels after he has landed.

After travelling through Croatia in 2014, I wrote:

I most enjoyed the activities where we weren’t charged an entry fee.

• Riding a scooter through the countryside of Hvar
• Swimming off the coast of the island of Vis, free to explore the coves and rock formations
• Aimlessly wandering the streets of ancient Croatian cities

When an entrance fee is charged, I felt a certain manicured, restricted, and “ushered” experience. Krka National Park— while attempting to showcase the natural beauty of the region, it failed because the experience was limited to a short walk along a wooden boardwalk. A long queue of tourists snaked around the park, each one experiencing the same thing, taking photographs on cue from the designated “Scenic Vista” points along the way.

It was a nature park that offered a menu of canned experiences devoid of anything approaching natural.

Entry fees and queues are a good sign that more authentic experiences can probably be found elsewhere. Freely wandering without a guide, without a destination, and without expectation can get you very far.

Its October 2016. I have the privilege of quitting my job at an Artificial Intelligence startup in Cambridge that I’ve been helping to build for the last four and a half years. I’m ready for some serious alone-time, away from computers, the internet, and people. I need time to myself, time to have unstructured thought, time to get outside my own skull. I’ve got a little more than two weeks of unscheduled time to tramp around from Italy into Switzerland.

This trip was about actually travelling between places on my own feet. Navigating terra incognita by push my own limits- setting a goal on the horizon and getting there by any means. My goal was to hike across the border from italy into Switzerland via the Italian/Swiss Alps.

Importantly, this trip wasn’t about setting off on a ‘Hero’s Journey’, to return home a changed man, but to spend time alone to better understand myself. It was a journey inwards. More than anything else, I had ample time to myself, whole days went by without talking to anyone.

“For my part, I travel not to go anywhere, but to go. I travel for travel’s sake. The great affair is to move; to feel the needs and hitches of our life more clearly; to come down off this feather-bed of civilization, and find the globe granite underfoot and strewn with cutting flints.”

I learned to be kind & gentle with myself, to calm my own anxieties when stressed, to assess reality with facts and realize only my actions will have any impact on the outcomes. To foster a feeling of contentment with what is, not what could or may be. These are things I discovered not out on the road, or on a mountain path, but within myself. These learnings are not discoverable by going farther, or hiking higher, but by allocating time and opening oneself to finding them. One has to both open the door, but then be willing to walk through the doorway.

Travel is just a big distraction of novelty and the remarkable. It’s easy to forget everything and be immersed in the distraction. In a world where we all become hyperspecialized in our careers, travel lets you multiply your experiences of life, hopefully getting a taste of other possible lives so you can tell yourself what it would’ve been like. Travel becomes the yearly experience people remember, the marker we live by when our lives become banal.

“Tourism, human circulation considered as consumption, a by-product of the circulation of commodities, is fundamentally nothing more than the leisure of going to see what has become banal. The economic organization of visits to different places is already in itself the guarantee of their equivalence.”

So if you are looking for something inwards, be prepared to focus on whats inside, not whats outside. Travel can be an escape from oneself or ones responsibilities, an escape more powerful than other forms of escape.

In the end, travel for self-improvement is snake-oil just as any other form of organized, packaged, advertised and productized self-improvement (e.g. self-help books, expensive Yoga classes, meditation get-aways, therapy sessions, magical potions and boner pills). The real work can’t be bought, it must be practiced- and you don’t need to leave the house to do it. Travel doesn’t change personal growth any more than puzzles improve intellectual growth.

1. Travel Writing - Redux, Italy & Switzerland Edition
2. Against Facebook
3. I am not a software developer
4. The Disappearance of Lived Time

For me, this means listening to music that doesn’t distract from the task of writing and problem solving. This set was designed in the spirit of musicForProgramming.net.

This set was heavily inspired by sci-fi movies that use long scenes to showcase the vast, open, and cold expanses of space: Solaris, Interstellar, and Gravity all come to mind.

Set List

• Tim Hecker - Harmony in Blue I
• Tim Hecker - Harmony in Blue II
• Nine Inch Nails - Harmony in Blue I
• Overand - Autechre
• Oscar See Through Red Eye - Boards of Canada
• Wolf Cub - Burial & Four Tet
• Zauberberg 6 - Gas
• Maybe You’re My Puppet - Cliff Martinez (Solaris)
• Wormhole - Cliff Martinez (Solaris)
• The Wormhole - Hans Zimmer (Interstellar)
• A Stop at the Chord Cascades - Tim Hecker
• Xerrox Helm Transphaser - Alva Noto
• Lonely Void - Mica Levi (Under the Skin)
• The Lisbon Maru - Fuck Buttons
• Along - Oneohtrix Point Never
• See on See - Autechre
• Hatred of Music I - Tim Hecker
• Hatred of Music II - Tim Hecker
• Carrying Glass - Ryuichi Sakamoto & Alva Noto

In following this guide, I’ve made some decisions that are entirely based on my own (subjective) opinions. These opinions are based on my experiences with other operating systems and distros, and a number of window managers I’ve tried.

My decisions (take them or leave them):

• Distribution: Arch Linux
• Full disk encryption: On
• Window Manager: Awesome
• Typography: infinality-bundle with the “Free” preset

This article assumes you’re currently running on a MacBook Pro between generation 7,1 (Mid-2010), 8,2 (Early-2011) or 11,3 (Late-2013). It assumes you’re running OS X and already have some experience with Linux commands, disk partitions and GNU/Linux. I’ve tested this guide on both of the above generations, but I’m assuming it’ll work on anything in between. However, I make no guarantees.

If you just want to get on to the installation, skip to this link: Installing Arch.

Contents:

• Background
  • Why Apple?
  • Why Arch?
  • Why Awesome?
• Preparing to Install Arch
  • Getting the Installation Media
  • Preparing the Installation USB Drive
• Installing Arch
  • Test Internet and Set the System Clock
  • Partition the Hard Drive
  • Configuring Drive Encryption and LVM
  • Select a Mirror
  • Install the Base System
  • Generate the fstab
  • Configure The System
  • Boot Loader
  • Reboot into New Installation
• Configuring Arch
  • Install an Arch User Repository Package Manager
  • Configure Sound
  • Install X and Video Drivers
  • Some GUI Applications
  • Improved Typography
  • Window Manager Awesome
  • Touchpad Support
  • Configuring Wireless
  • Done!
• Fine Tuning
  • Display Color Profile
  • Display Color Correction
  • Power Management
  • Laptop Mode Tools
  • acpid
  • CPU Frequency Scaling
  • Temperature Management
  • Fan Control
  • Apple Keyboard
  • Apple Trackpad
• Conclusion
• References
• Footnotes

This article will be installing Arch Linux alongside OS X, dual-booting such that you can easily boot into either. I recommend this, even if you’re not going to be using OS X at all, because right now (and likely for the foreseeable future) the only way to get firmware updates installed on your machine will be via OS X. There have been Linux kernel–MacBook Pro firmware compatibility issues in the past that have been fixed by OS X updates that installed firmware fixes.

I assume you have already repartitioned your drive giving you plenty of free space for Arch. On my 500GB drive, I left 80GB for OS X (probably much more than really necessary) and the rest I left free for Arch.

Conventions

For commands typed as the normal user, I will not prefix them:

uname -a

For commands that need to be executed as a root account, i will prefix them with a hash-mark: #

# uname -a

Background

Why Apple

I’ve been using OS X as my primary OS since 2011 when I bought my first Apple product, the 15” MacBook Pro 8,2− which I still use today (March 2016, on which I’m writing this article). This alone is a testament to its’ longevity as a computing platform. The industrial design of the hardware is a pleasure to the eyes and is much imitated. The hardware feels wonderful to the touch. I’ve disassembled a few of them, and despite the security screws, they’re fairly easy to repair or upgrade. Only recently had I felt that its’ age was starting to show, so last week I doubled the ram from 8GB to 16GB and upgraded the hard drive to a 512GB SSD. This should provide at least another two years of life for this hardware.

Recently however, Apple has changed their hardware design philosophy from a fairly open platform to a more proprietary and disposable one. They’ve eliminated the ability to change or upgrade components. The 3rd generation MacBooks have a battery that is glued in place, the memory is soldered directly to the logic board. Later models would come with the hard drive soldered down as well.

Lyle Wiens of iFixIt, said it best in 2012

When Apple dropped the MacBook Air to $999 in 2010 to match the price point of the MacBook, they gave users a clear choice: the thin, light, and un-upgradeable MacBook Air or the heavier, longer lasting, [upgradable], more rugged, and more powerful MacBook. Same price, two very different products. At the time, I wasn’t very happy with the non-upgradeable RAM on the MacBook Air, but I respected that Apple had given their users a choice. It was up to us: Did we want a machine that would be stuck with 2GB of RAM forever? Would we support laptops that required replacement every year or two as applications required more memory and batteries atrophied?

Consumers overwhelmingly voted yes, and the Air grew to take 40 percent of Apple’s notebook sales by the end of 2010.

This sort of vendor-lock down and planned-obsolecence has bothered me to such an extent that the next laptop I buy won’t be Apple hardware.

Meanwhile, Apple has secured an oligopoly in the market, they have such a dominant position with hardware manufacturers that they seem to be squeezing competitors out of getting access to top-quality components (or their competitors don’t care about quality). These manufacturers seem to be building computers with “Apples’ scraps and leftovers”.

Put simply; I don’t see any manufacturers building laptops at the same level of quality that Apple does, and yet, I can’t buy a product¹ that is so tightly controlled. Finally, I refuse to buy a product that is designed to be obsolete in two years.

Why Arch?

I first started using Linux in 1997 when I bought a shrink-wrapped box of RedHat Linux 5.2. At the time, my internet connection was fast but not reliable enough to download the ISO image over a 1-day dial-up session. Since then I’ve tried Slackware, SuSE, Mandrake, Debian, Fedora, Ubuntu, CentOS, and Mint. I customized the hell out of my distros, I performance tweaked, secured and customized the GUI. Back then, information was hard to come by, the community was still small and poorly documented, and that hardly mattered when I couldn’t access the internet because my ethernet card drivers wouldn’t compile.

Jen Andre sums it up best

You kids these days. You have it easy.

It was enjoyable for a while, but I got very busy and eventually came to the conclusion that “Linux is free if your time is worth nothing” and stuck with OS X. I would occasionally install and try a different distro just to see if the OSS operating system world had changed, but that was about it.

Arch is different however. Arch gets it, Arch gets it right. Arch is what I was searching for all those years.

1. A rolling release cycle keeps everything fresh
2. I’m a huge advocate for their KISS Principal: Keep it Simple…
3. The documentation is widely regarded as the best source of info on GNU/linux
4. The Core package repository is very well maintained and trusted
5. The User package repository is great at providing pretty much everything you could need or want
6. Its designed for people who want to understand and build a system from the ground up. Install only what you know, only what you understand. It doesn’t hide anything behind assumptions, scripts or fancy installers.

Arch is like a sandbox, they provide the box and a source of sand (the Core and User package repos), but it’s up to you to fill it and build your castle.

Why Awesome?

Like the topic of GNU/Linux distros, I could spend an entire post talking about why I settled on Awesome. I’ve tried GNOME, KDE, even Enlightenment. They all are trying to do more than I need, are too influenced by the design of the Big Desktop players and pander to novices at the expense of the power users.

All I really need is a quick way to switch between windows and resize them as needed and for it to be fast. The focus of the window manager is the application contents, not the chrome eye-candy around the edges of the window.

Awesome was designed for this.

So, this guide is going to install Awesome as it’s window manager. Feel free to switch it out for whatever you prefer. i3, xmonad and DWM are all similar in design.

Preparing to Install Arch

When doing this article, I found many other articles very useful. They are listed in References. Please refer to them if needed. I will attempt to both update and condense these articles to a single “Be-all End-all Guide” to MacBook installation.

Getting the Installation Media

First, we need to get Arch Linux ISO from Arch Linux, Go here to download it

I usually go with the torrent option. It’s the fastest.

Next, we’re just going to make sure the download wasn’t corrupted or tampered with in transit. To make this possible, the Arch Linux ISO publisher has posted a cryptographic hash of the ISO. We can compare the hash of the ISO they published against the hash of the ISO we downloaded. If the hash is identical, we know the contents are exactly the same. Start the terminal and run the following:

sha1sum <FILE>.iso

Next, we want to make sure the the ISO we’ve downloaded is provably supplied by the Arch Linux team. We can do this by verifying the cryptographic signature they provided with the ISO. The Arch team cryptographically signs their ISO images using public-key cryptography. This ensures that the ISO file we’ve downloaded is verifiably provided by the Arch Linux team, and not an impostor.

This assumes you have GnuPG installed on your system (brew install gnupg). On the ISO download page, there is a link under “Checksums” to get the PGP signature as a .sig file. Download that into the same directory as your Arch ISO. Then run the following, replacing the filename (here i use archlinux-2016-02-01-dual.iso) with the name of the files you’ve downloaded:

gpg --verify \
    archlinux-2016.02.01-dual.iso.sig \
    archlinux-2016.02.01-dual.iso 2>&1 | \
    grep 'key ID' | \
    gpg --recv-keys 2>&1 `awk '{print $NF}'` && \
gpg --verify archlinux-2016.02.01-dual.iso.sig \
    archlinux-2016.02.01-dual.iso 2>&1 | grep 'signature from'

This will first attempt to verify the signature, and if you don’t have the signers key, it’ll retrieve it. It’ll then try to verify it again, and should print out “Good signature” if it succeeds. If you get “Bad signature” this means the ISO has either been forged by an impostor pretending to be the Arch linux team, or someone has tampered with the ISO stored on their servers or in transit. If thats the case, you need to find an alternate source to download the ISO.

Preparing the Installation USB Drive

This installation method will first create a bootable USB stick, which is used to boot into a “Live” Arch Linux session. From there, we use the Live Arch Linux to install Arch onto your MacBook hard drive, then make that partition bootable.

If you’re currently running a GNU/Linux system, follow the instructions in Part A. If you’re running OS X, jump one section ahead to Part B.

Part A - Preparing the Installation USB Drive for Linux

If you’re already on a GNU/Linux system, use these instructions to create USB bootable Arch Linux live system.

Now we’re ready to get that ISO onto a USB drive so we can boot the computer from it. Before inserting the thumb drive, run lsblk and take note of the drives listed. Insert the drive and run lsblk again. Take note of the new letter in the sdX list of drives.

Replace the X with the letter on your system. Be sure to unmount the new drive.

umount /dev/sdX

Next, run the following command, it reads the ISO file you downloaded earlier and writes the contents directly to the USB drive, without the operating system buffering the writes. Replace the X with the letter of the USB stick you took note of earlier, and ARCHLINUX with the name of the image file you downloaded.

dd if=ARCHLINUX.iso of=/dev/sdX bs=4M

After that’s run, the USB drive should be ready to boot from.

Part B - Preparing the Installation USB Drive for OSX

Now we’re ready to get that ISO onto a USB drive so we can boot the computer from it. Before inserting the thumb drive, run diskutil list and take note of the drives listed. You should see output similar to this:

    /dev/disk0
       #:                       TYPE NAME          SIZE       IDENTIFIER
       0:      GUID_partition_scheme               500.1 GB   disk0
       1:                        EFI               209.7 MB   disk0s1
       2:          Apple_CoreStorage               399.5 GB   disk0s2
       3:                 Apple_Boot Recovery HD   650.0 MB   disk0s3
       5:                 Apple_Boot Boot OS X     134.2 MB   disk0s5
    /dev/disk1
       #:                       TYPE NAME          SIZE       IDENTIFIER
       0:                  Apple_HFS MacOSX        399.2 GB   disk1

Insert the drive and run diskutil list again. Take note of the new letter in the sdX list of drives.

    /dev/disk0
       #:                       TYPE NAME          SIZE       IDENTIFIER
       0:      GUID_partition_scheme               500.1 GB   disk0
       1:                        EFI               209.7 MB   disk0s1
       2:          Apple_CoreStorage               399.5 GB   disk0s2
       3:                 Apple_Boot Recovery HD   650.0 MB   disk0s3
       5:                 Apple_Boot Boot OS X     134.2 MB   disk0s5
    /dev/disk1
       #:                       TYPE NAME          SIZE       IDENTIFIER
       0:                  Apple_HFS MacOSX        399.2 GB   disk1
    /dev/diskX
       #:                       TYPE NAME          SIZE       IDENTIFIER
       0:      GUID_partition_scheme               2.0 GB     disk2
       1:       Microsoft Basic Data UNTITLED 1    2.0 GB     disk2s1

Note /dev/diskX has appeared (‘X’ will be a number on your system), this new disk is the USB thumb drive. Important take note of the number of this drive.

We’re going to remove the existing partitions and erase all the data so its clean for our Arch Linux installer. This will delete all the data on the USB drive. Make sure to substitute /dev/diskX for the drive number you noted above.

diskutil partitionDisk /dev/diskX 1 "Free Space" "unused" "100%"

Now² we can write the iso file to the USB drive. Note We use /dev/rdisk* instead of /dev/disk because it provides Raw disk access without the typical buffering the operating systemprovides. Substitute DESTINATION with the name of the iso file you downloaded earlier, and substitute the X with the number of the drive.

dd if=DESTINATION.iso of=/dev/rdiskX bs=1m

The dd command does not show any output before it has finished the copy process, so be patient and wait for it to complete, it can take around 1 - 5 minutes. When the command does complete OS X will try to mount the drive and fail as it won’t recognize the formatting. Click ignore.

Installing Arch

With the USB drive plugged in, restart the MacBook and press+hold down the left option/alt key when you hear the startup chime. Hold the option key down until the drive screen appears.

The USB drive should be one of the options. Pick it along with the first option the following boot screen lists. Now you should be at the Arch live install prompt and it’ll look like this:

    Arch Linux 3.17.6-1-ARCH (tty1)

    archiso login: root (automatic login)
    root@archiso ~ #

You are now running a “Live” session of Arch Linux from the USB stick.

If you’re on a HiDPI / “Retina” display MacBook, the prompt is really small. Increase the font-size using setfont sun12x22.

Test Internet

You’ll need a working Internet connection to do the post-install. My MacBook Pro 7,1 and 8,2 both have Ethernet ports, but some of you will have later models without one. You should get a USB-Ethernet adapter. Its much easier than fiddling with wireless drivers as it works out of the box. We’ll setup the wireless drivers later in this article.

First, we need to get an IP address from your router:

dhcpcd

With the adapter plugged in and an IP address, make sure internet is flowing by pinging Google:

ping -c 3 www.google.com

You should get a response that all three packets were sent and received.

Set the System Clock

The system clock should be just fine. The general Arch wiki recommends ensuring the system clock is accurate, and I’ve found it doesn’t seem to break anything. So, I run it:

timedatectl set-ntp true

Partition the Hard Drive

A partition is basically a box on the hard drive to put files into. OS X has a box for its files, and we’re going to create box(es) for GNU/Linux. Partitioning the hard drive takes existing data on the drive and moves or erases it to make space available for other file systems.

This part is the trickiest because it can erase data on the disk, so take care.

Apple uses the GUID Partition Table (GPT), and we’re going to keep that partition table.

In order to proceed you’ll need to know the drive mapping scheme, we’ll use fdisk to check the scheme:

fdisk -l

This lists the existing partitions. If you created two partitions when preparing the MacBook, you should see a partition with a Type of Apple HFS/HFS+ with a size that matches the size you set aside for Arch. In my case this was /dev/sda5. All the partitioning commands below will use /dev/sda5, you should substitute the designation for your drive.

Lets use cgdisk to view the partition table on the attached devices: Replace the Y with the drive you’ll be using to install Arch Linux.

cgdisk /dev/sdY

At the end of the partition table should be the free space you set aside for installing Arch. With that space, we’re going to create a new partition. You just need to make one partition; we’re going to break it into sub-partitions later; after encryption is setup.

I added 128MB between the last partition and my new partition because of this explanation by Apple.

Note: We leave free space after each partition to make it easier for future system software to manipulate the partition map in ways that we can’t anticipate currently.

Begin by deleting the HFS partition created when you set aside space in OS X for Arch Linux.

In order to leave additional space, just type +128M when you create the new partition, and it’ll set the starting sector at a point 128M away from the ending sector of the partition before it.

You’ll want to use Linux LVM (8e00) as the partition type id. The final partition table will look something like this:

    Part. #     Size        Partition Type            Partition Name
    ----------------------------------------------------------------
    3.0 KiB      free space
    1        200.0 MiB      EFI System                EFI System Partition
    2         74.5 GiB      Apple HFS/HFS+            Macintosh HD
    128.0 MiB      free space
    3        391.1 GiB      Linux LVM                 ArchLinux

Then use the utility to select Write and then confirm that you want to overwrite the disk. Once the display returns you can Quit.

Running fdisk -l again should show your new partition scheme. If it doesn’t look like you want or expected, repeat the cgdisk process to fix things until you’re satisfied.

Configuring Drive Encryption and LVM

You’ll want to make note of the partition number you just created. For me it’s partition 5, and the drive is sda, so my Arch Linux partition can be found at /dev/sda5. I’ll be using that as an example going forward, but you should substitute your own drive path.

We’re going to encrypt /dev/sda5 using DM-Crypt and then the LVM partitions are going to be created over that LUKS encryption layer. This system is called “LVM on LUKS.” Both LUKS encryption and LVM support are provided by the GNU/Linux kernel.

Note This will just encrypt the system / and /home directories. The /boot directory will not be encrypted because we’re going to keep the existing /boot partition: That’s /dev/sda1 in my partition table above.

If you want to use custom ciphers, there are some great notes available on GitHub. You can also use the following command for speed benchmarking to help determine which ciphers and key-sizes are fast enough for your particular use case:

cryptsetup benchmark

Time to pick the encryption flavors! The default values for the cipher and key sizes were a bit too light for my tastes (in light of the NSA spying scandal). I’ve increased these numbers above the defaults, I’ve chosen to balance my principals for privacy and security with practical usability and speed. Feel free to read more on these settings.

cryptsetup --cipher aes-xts-plain64 \
  --key-size 512 \
  --hash sha256 \
  --iter-time 3000 \
  --use-random \
  --verify-passphrase \
  luksFormat /dev/sda5

Enter in a good passphrase (twice), and we should be good to go.

Now with the encryption setup, we’re going to create the necessary volumes and filesystems within the LVM.

First, let’s open up our encrypted partition:

cryptsetup luksOpen /dev/sda5 lvm

This is going to map our encrypted device (/dev/sda5 in my case) to /dev/mapper/lvm. Now we’re going to create the physical and logical volumes for the / and /home directories. I gave the / directory 40GB (hopefully enough for all my programs. As of writing this, and with a full install, I’m using 10GB on my / directory. So, I think I’m good.

Create the physical volume:

pvcreate /dev/mapper/lvm

Now create the volume with the name vgcrypt:

vgcreate vgcrypt /dev/mapper/lvm

We’re ready to create the logical volumes now, 40GB for root and the rest for users’ home, change 40GB accordingly:

lvcreate --size 40G --name root vgcrypt
lvcreate --extents +100%FREE --name home vgcrypt

We now have our two volumes vgcrypt-root and vgcrypt-home. They need to be formatted to a particular filesystem. I’ve been happy with the ext4 filesystem.

mkfs.ext4 /dev/mapper/vgcrypt-root
mkfs.ext4 /dev/mapper/vgcrypt-home

We can now mount these new partitions. Make sure to mount the root partition first so we can create the /home directory inside of it for the home partition:

mount /dev/mapper/vgcrypt-root /mnt
mkdir -p /mnt/home
mount /dev/mapper/vgcrypt-home /mnt/home

Lets also mount our boot partition, while we’re here. This is required so our bootable initramfs can be written to the boot drive:

mkdir -p /mnt/boot
mount /dev/sda1 /mnt/boot

If you’re interested in swap partition schemes, check this script out − although I’m fine without swap partitions (Its faster with enough RAM).

And with that, Arch is now ready to be installed on the disk.

Select a Mirror

This step can optionally be skipped, but I prefer to choose a US server just in case it may be faster. Open up the mirrorlist:

vi /etc/pacman.d/mirrorlist

Delete or comment out all the servers except one or two near you that seem good.

Install the Base System

Actually installing Arch Linux onto the drive is the easiest part:

pacstrap -i /mnt base base-devel terminus-font

The -i flag asks for confirmation before installing packages. I like using it just so I can see what’s being installed. (After all that’s part of the reason for using GNU/Linux, right? To know what’s being installed on your system.)

This installation includes terminus-font, which we’ll configure to be the default console font later on.

Generate the fstab

If all went according to plan, Arch has been written to the hard drive and is now installed. Before rebooting into our installation, though, we need to tell the system where to find the filesystems we created earlier for root and home directories.

While it’s normally a good idea to use UUIDs to find disks, we’re going to use labels. This is because our encryption setup generates random IDs for the disks when they’re decrypted. Let’s create the fstab file:

genfstab -L -p /mnt >> /mnt/etc/fstab

The -L flag will generate the fstab file with labels instead of UUIDs. The -p flag prevents pseudo-filesystems from being added.

Always check the generated fstab:

cat /mnt/etc/fstab

It should look something like this:

#
# /etc/fstab: static file system information
#
# <file system>    <dir><type><options>             <dump><pass>
# /dev/mapper/vgcrypt-root
/dev/mapper/vgcrypt-root    /           ext4   discard,rw,relatime,data=ordered    0 1

# /dev/mapper/vgcrypt-home
/dev/mapper/vgcrypt-home    /home       ext4   discard,rw,relatime,data=ordered    0 2

Note If your hard drive is a solid-state drive (SSD) and the discard option isn’t there, edit the fstab file and add it. It’s used for optimizing for the speed of SSDs.

Configure The System

We’re now ready to configure our new system. Let’s change root into it:

arch-chroot /mnt /bin/bash

Set our system locale. I’m in the US, so I’ll only uncomment that locale from /etc/locale.gen:

...
en_US.UTF-8 UTF-8
en_US ISO-8859-1
...

Now generate the locales:

locale-gen

Make English UTF-8 the default:

echo LANG=en_US.UTF-8 > /etc/locale.conf

The default font in the virtual console is not very readable, so lets use one that is far more readable. We’re going to use the typeface “Terminus” in size 18 that we installed with the base system earlier. ter-118n basically means “Terminus latin-1 size 18 Normal”. There are other sizes available: 12, 14, 16, 20, 22, 24, 28, 32 as well as support for multiple non-English code pages.

echo FONT=ter-118n > /etc/vconsole.conf

Set the timezone accordingly. (I live on the east coast):

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime

And set the time to the standard UTC:

hwclock --systohc --utc

Because we’ve encrypted our root disk, we need to make sure the kernel loads the proper modules to decrypt it on startup. Otherwise we won’t be able to boot from the unencrypted drive. We’re also going to tell the boot-sequence to load our custom font and keyboard module so we can type our passwords during boot. Edit /etc/mkinitcpio.conf and add the hooks consolefont keyboard encrypt lvm2 BEFORE filesystems so the HOOKS line looks like this:

HOOKS="base udev autodetect modconf block consolefont keyboard usbinput encrypt lvm2 filesystems fsck"

Now we need to regenerate the initramfs image:

mkinitcpio -p linux

So that Internet will work on reboot, we need to enable the dhcpcd service. We’re going to keep using the ethernet-USB adapter for right now. We’ll get wireless setup later. Get the name of the ethernet interface:

ip link

It should be enp-something. With that, enable the service, make sure you replace INTERFACE with your interfaces name.

systemctl enable dhcpcd@INTERFACE.service

Finally, let’s configure the machine’s hostname. You can change macbook to whatever you’d like:

echo macbook > /etc/hostname

Add this hostname to the list of hosts. Edit /etc/hosts and edit so it looks something like this:

#
# /etc/hosts: static lookup table for host names
#

#<ip-address>   <hostname.domain.org>   <hostname>
127.0.0.1   localhost.localdomain   localhost   macbook
::1     localhost.localdomain   localhost   macbook

Set a root password:

passwd

Create a non-root user for yourself and set the user’s password, replace USER with the username of your choosing:

useradd --create-home --groups wheel --shell /bin/bash USER
passwd USER

This will create the user, add it to the group wheel (traditional group of users who can run sudo commands), create a home directory under /home/USER/ and make his default shell bash then set a default password for the user.

Don’t switch to the new user yet, because we’e added USER to the group wheel, lets grant the wheel group sudo privileges. Run:

visudo

And uncomment the following line so it looks like so:

%wheel ALL=(ALL) ALL

Boot Loader

Now we need to let the boot loader know where to find our new Arch Linux installation. systemd (was recently renamed, was Gummiboot) is a nice, simple boot loader.

# pacman -S systemd              # May be systemd-boot, included in core?
mkdir -p /boot/loader/entries

Setup the loader to default to arch and set the number of seconds to timeout in the file /boot/loader/loader.conf:

default arch
timeout 3

Make sure the correct boot partition (/dev/sda1 in my case) is mounted on /boot by running:

findmnt /boot
-------------------------
TARGET SOURCE    FSTYPE OPTIONS
/boot  /dev/sda1 vfat   rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,err

Now we’re going to create an entry called /boot/loader/entries/arch.conf that looks like this:

# /boot/loader/entries/arch.conf
title   Arch Linux
linux   /vmlinuz-linux
initrd  /initramfs-linux.img
options cryptdevice=/dev/sdaX:vgcrypt:allow-discards root=/dev/mapper/vgcrypt-root rw

Note:³ On the options line, make sure you replace /dev/sdaX with the path to the encrypted linux partition we previously created on your device (It was /dev/sda5 in this guide). Additionally, if your drive is not a SSD, make sure to remove :allow-discards.

Note:⁴ Its been reported by some users that when booting to USB, the USB drive is assigned device id /dev/sda and the hard drive to /dev/sdb. When rebooting without USB drive (boot to new encrypted linux drive partition), the hard drive is reassigned to device id /dev/sda which may invalidate the above arch.conf boot loader entry. In this case, the encrypted linux partition will not boot because the boot loader config arch.conf is configured to boot from /dev/sdb. In this case you can reboot to USB again, and edit the options line in arch.conf to read /dev/sdaX. Replace X with your partition id.

Check the boot tree with tree /boot/ (If tree isn’t installed, install it with pacman -S tree). It should look something like this:

/boot/
├── EFI
│   ├── APPLE
│   │   └── EXTENSIONS
│   │       └── Firmware.scap
│   ├── Boot
│   │   └── BOOTX64.EFI
│   └── gummiboot
│       └── gummibootx64.efi
├── initramfs-linux-fallback.img
├── initramfs-linux.img
├── loader
│   ├── entries
│   │   └── arch.conf
│   └── loader.conf
└── vmlinuz-linux

Note: At this point, you may want to enable suspend-to-disk (No power consumption aka “Hibernate” mode). I’ve chosen not to enable this functionality, but if you’re interested, you can read about it here.

Now we can have the bootloader write the initramfs onto the boot partition using bootctl, part of the systemd-boot package.

bootctl install

Reboot into New Installation

Let’s leave the chroot environment we used for the install:

exit

You can umount and close the encrypted volume:

umount -R /mnt
cryptsetup close vgcrypt

It’s not a bad idea to just double check the encryption to make sure it opens and mounts properly:

cryptsetup open /dev/sda5 vgcrypt
mount /dev/mapper/vgcrypt /mnt

If all goes accordingly, you can unmount and close the encryption again. And finally…

reboot

On reboot you should be greeted with the boot menu. After selecting Arch (or waiting for it to timeout), you should be prompted for your password to decrypt the drive, and then it should boot into the console.

Configuring Arch

You should now be running Arch Linux! Lets customize it and make it more useful.

Install an Arch User Repository Package Manager

Now we’re about to dive deep into customizing Arch Linux installation with the utilities and application we’ll be using on a day-to-day basis. To facilitate installing packages from the wider Arch User Repository (AUR), we’ll install a utility named yaourt. You can (of course) maintain AUR package installations manually, but I like a helper to help me manage them. yaourt is like the Homebrew of Arch Linux. I use yaourt because it’s easy to install, takes the same optional flags as pacman, and works well.

You need to use the “official” and manual way of installing AUR packages to get yaourt installed. It also requires package-query from the AUR, so we’re going to install that first.

Now we’re going to download the package-query AUR package, unarchive it, and use pacman to build and install the package. This allows us to manage the package as if it was installed with pacman, although we’re manually installing it.

$ cd ~
$ curl -L -O https://aur.archlinux.org/cgit/aur.git/snapshot/package-query.tar.gz
$ tar -xvzf package-query.tar.gz
$ cd package-query

Within the packages’ folder, we’re going to run the following to build it:

$ makepkg -s

This will run as your regular user only asking you for your root password if necessary. With the package made, we can install it via pacman:

# pacman -U package-query-1.6.2-1-x86_64.pkg.tar.xz

Now we’ll do the same for yaourt:

$ cd ~
$ curl -L -O https://aur.archlinux.org/cgit/aur.git/snapshot/yaourt.tar.gz
$ tar -xvzf yaourt.tar.gz
$ cd yaourt
$ makepkg -s
# pacman -U yaourt-1.6-1-any.pkg.tar.xz

From now on you can use pacman and yaourt interchangeably… for the most part. yaourt will install Arch repository packages and AUR packages. pacman, though, will continue to only install Arch repository packages.

Configure Sound

ALSA works out of the box with Macs so install it via:

# pacman -S alsa-utils

Then use:

alsamixer

to control the speakers. Make sure to disable channels for speakers you don’t have. Test your speakers with:

speaker-test -c 2

where 2 is the number of speakers.

Install X and Video Drivers

As this is going to be a graphical interface, we need are graphics card up and running with the proper drivers. Luckily this machine uses Intel graphics, and Intel is pretty good about providing Linux (and sometimes open-source) drivers.

For Macbookpro 7,1 (Mid-2010 or 8,2 (Early-2011):

# pacman -S xf86-video-intel mesa-libgl libva-intel-driver libva

For Macbookpro 11,3 (Late-2013)⁵

# pacman -S nvidia mesa-libgl libva-intel-driver libva

The -S flag tells pacman to install the subsequent packages listed. (Again, you can use yaourt if you’d like.) This will install the Intel video driver, the Mesa OpenGL graphics library, and video acceleration API for Intel graphics chipsets. It’ll likely ask you to install additional dependencies. Get all the dependencies!

With the necessary drivers installed, we can get Xorg (or the X Window System) installed.

# pacman -S xorg-server xorg-server-utils xorg-xinit xterm

I like to install the Xorg utilities, too, because there are at least a couple I’ll use later (either in this guide or another) that are helpful in improving HiDPI support for the MacBook’s HiDPI Retina display.

Some GUI Applications

Next we’ll install a browser. This will install some typeface dependencies and font- rendering libraries that we’ll be tweaking later.

# pacman -S firefox

Improved Typography

Update⁶

It appears the package maintainer for Infinality has gone dark. Skip this section and continue to the next until there is a solution for better typography on the linux desktop.

If you use the web and appreciate typography, you’ll want to install a set of decent fonts.

Infinality is a package group that has been meticulously crafted from the ground-up to provide beautifully rendered typography to the Linux platform. The package maintainer spent a ton of time tearing down the existing font stack, and carefully rebuilding it to provide fast-rendering fonts that more faithfully present the typography than either Windows, OS X or Ubuntu systems. It also provides a set of preset-configurations that allow the user to switch to a Windows-like or OS X-like configuration. I highly recommend reading the documentation if you’d like to know more about typography rendering on GNU/Linux.

First add the following package repositories to /etc/pacman.conf.

    [infinality-bundle]
    Server = http://bohoomil.com/repo/$arch
    [infinality-bundle-fonts]
    Server = http://bohoomil.com/repo/fonts

Next we’ll add the package maintainers PGP key to the package database, update the package database and install the packages.

# pacman-key -r 962DDE58
# pacman-key --lsign-key 962DDE58
# pacman -Syyu
# pacman -S infinality-bundle
# pacman -S infinality-bundle-fonts

Make sure to select the option fontconfig-infinality-ultimate as that configuration is the most clean, efficient and looks the best out of the box. The installation may ask to use the Infinality packages in place of the dependencies installed by Firefox. Choose the Infinality packages.

To enable selection of predefined font substitution styles and antialiasing settings, apart from the rendering settings of the engine itself. After doing so, you can select the font style (win7, winxp, osx, linux, …) with:

# fc-presets set

Window Manager Awesome

Awesome is a tiling window manager for the user that prefers keyboard commands to mouse or touchpad, and treats window chrome as needless ornamentation. Your application content is king, and efficiency is the goal. Its quick and light. Lets install the window manager (WM), awesome.

To install:

# pacman -S awesome

Added this⁷ to run awesome when x starts:

echo exec awesome > .xinitrc

Touchpad Support

You’ll probably want to use your MacBook’s touchpad when you have a GUI. The simplest driver is the synaptics driver:

# pacman -S xf86-input-synaptics

The following config at /usr/share/X11/xorg.conf.d/70-synaptics.conf works well for me, it uses the same “Natural Motion” that OS X does. Copy that file to the following location /etc/X11/xorg.conf.d/70-synaptics.conf and add the options between the START and END comments.

Section "InputClass"
    MatchIsTouchpad "on"
    Identifier      "touchpad catchall"
    Driver          "synaptics"

    # START: Add these options
    # 1 = left, 2 = right, 3 = middle
    Option          "TapButton1" "1"
    Option          "TapButton2" "3"
    Option          "TapButton3" "2"
    # Palm detection
    Option          "PalmDetect" "1"
    # Horizontal scrolling
    Option "HorizTwoFingerScroll" "1"
    Option "VertTwoFingerScroll" "on"
    # Natural Scrolling (and speed)
    Option "VertScrollDelta" "-450"
    Option "HorizScrollDelta" "-450"
    # END
EndSection

Configuring Wireless

Before rebooting into our lovely new GUI, let’s get wireless setup to work when we reboot.

This particular machine has the Broadcom BCM4360 wireless chipset. Broadcom has been pretty mixed in the FLOSS support it seems. The BCM4360 is not supported by the kernel itself, so we’ll need to use Broadcom’s non-free, non-open driver: broadcom-wl. I actually don’t think this even officially supports the BCM4360 chipset, but it works well enough. We’ll need to install the AUR package:

$ yaourt -S broadcom-wl dialog wpa_supplicant

And activate the kernel module:

# modprobe wl

Note: If you update to a newer kernel in the future, you may need to uninstall and reinstall the broadcom-wl package so it updates with the new linux-header.

We need to stop the dhcpcd service we were using for the ethernet and start the wifi-menu¹ utility. Keeping both running can cause conflicts.

# systemctl disable dhcpcd.service
# wifi-menu

This will create and enable a systemd service that will start when the computer boots. Changes to the profile file will not propagate to the service file automatically. After such changes, it is necessary to reenable the profile:

# netctl reenable PROFILE

After enabling a profile, it will be started at next boot.

Finally, if you find your wireless is dropping connections, you may find turning off Wi-Fi power management. Simply create this as an executable ( chmod +x /etc/pm/wireless and add the following contents:

#!/bin/sh
iwconfig wlp2s0 power off

Done!

Now when you reboot, you should be all set to go start customizing to your heart’s content, adding applications as you’d like, and playing around with your new Arch Linux box with Awesome.s

Fine Tuning

Power settings took quite a bit of tweaking. Without these setting, the laptop ran very hot, and drained battery life very fast. I would recommend following these steps to improve battery life. There still may be room for improvement, though.

One more thing: If you search “MacBook Pro and Arch Linux” you’ll probably read some things about disabling ACPI interrupts that were causing overheating and high CPU usage. If you have updated to the latest version of OS X 10.10 “Yosemite” then you should be OK to skip.

Display Color Profile

I noticed that colors seemed washed-out in Arch Linux, so this task will attempt to color-correct the display. Luckily a utility xcalib exists that can load color profiles from OS X as X options. This should give you true color-parity between operating systems.

First, boot into OS X and copy the color profiles located in /Library/ColorSync/Profiles/Displays to somewhere so you can boot into GNU/Linux and copy them into your home directory. I created a directory named ~/.colorprofiles for these files.

Second, install the xcalib package from the Arch AUR.

yaourt -S xcalib

Finally you can activate it by running:

xcalib ~/.colorprofiles/FILENAME.icc

Where FILENAME is the path to your color profile file.

To load this color profile when X starts, I also added this command to the .xinitrc file in my home directory. Just make sure you replace FILENAME with the name of your color-profile .icc file exported from OS X.

if [ `type -P xcalib` ]; then
  # Use the color profile
  xcalib ~/.colorprofiles/FILENAME.icc
fi;

Display Color Correction

If you spend as many hours looking at the display as I do, you’ll quickly appreciate a display that adapts its colors to the time of day. F.lux adjusts monitor color temperature adaptively to ease eye strain.

yaourt -S xfluxd

Then edit /etc/xfluxd.conf to set your rough lat/long coordinates (in decimal format) in order to correctly shift the color correction with the sunrise and sunset.

Finally, enable and start the xfluxd service. Note you should run this as your normal user, not as root.

systemctl enable --user xfluxd
systemctl start --user xfluxd

Power Management

The power management from Arch out of the box is not very good. There are a few good tools out there, but PowerTOP is nice because of its benchmarking utilities⁸.

PowerTOP is a tool provided by Intel to enable various powersaving modes in userspace, kernel and hardware. It is possible to monitor processes and show which of them are utilizing the CPU and wake it from its Idle-States, allowing you to identify applications with particular high power demands.

yaourt -S powertop

You may want to put your laptop on battery power and calibrate powertop:

# powertop --calibrate

That’ll cause the screen to blackout from time to time. Just let it run. It takes a few minutes then your screen will come back on.

You can create a systemd service that will launch powertop’s autotune settings on startup.

# /etc/systemd/system/powertop.service

[Unit]
Description=Powertop tunings

[Service]
Type=oneshot
ExecStart=/usr/bin/powertop --auto-tune

[Install]
WantedBy=multi-user.target

And enable it to automatically start at boot time, then start it for your
current boot session.

# systemctl enable powertop.service
# systemctl start powertop.service

Laptop Mode Tools

Laptop Mode Tools⁹ is a laptop power saving package for Linux systems. It is the primary way to enable the Laptop Mode feature of the Linux kernel, which allows you to tweak a number of other power-related settings using a simple configuration file. Combined with acpid and CPU frequency scaling, LMT provides a complete notebook power management suite.

$ yaourt -S laptop-mode-tools

If you want to enable laptop mode even on AC power, because you run the laptop attached to an external keyboard and monitor, edit:

# /etc/laptop-mode/laptop-mode.conf`:
...
ENABLE_LAPTOP_MODE_ON_AC=1
...
ENABLE_LAPTOP_MODE_WHEN_LID_CLOSED=1

We’re going to disable LMT from handling CPU frequency scaling since we’ve setup cpupower to handle that:

# /etc/laptop-mode/conf.d/cpufreq.conf
...
# CONTROL_CPU_FREQUENCY="AUTO"
CONTROL_CPU_FREQUENCY=0

and disable Intel pstate handling as well:

# /etc/laptop-mode/conf.d/intel_pstate.conf
# CONTROL_INTEL_PSTATE="auto"
CONTROL_INTEL_PSTATE=0

Finally, enable and start the systemd service:

# systemctl enable laptop-mode.service
# systemctl start  laptop-mode.service

acpid

A very useful tool is from the acpi package, which provides battery information using the command acpi -v. To install:

# pacman -S acpi

acpid is an extensible daemon for handling ACPI events. It can run commands when the laptop lid is closed, etc.

# pacman -S acpid
# systemctl enable acpid.service
# systemctl start  acpid.service

CPU Frequency Scaling

Another utility that will help with CPU frequency scaling is cpupower, it provides useful CLI utilities and a systemd service to change the CPU governor at boot.

$ yaourt -S cpupower
# systemctl enable cpupower
# systemctl start cpupower

I use cpupower to modulate the CPU’s speeds. This keeps CPU in check from maxing out at all times. My cpupower config file at /etc/default/cpupower has the following line changed:

governor='powersave'

You should adjust this setting to your own needs.

Temperature Management

Intel provides a daemon that will keep tabs on the CPUs’ temperature and adjust settings to keep it from getting too hot, its called thermald.

$ yaourt -S thermald
# systemctl enable thermald.service
# systemctl start thermald.service

Fan Control

Finally, the kernel doesn’t seem to have very fine control over the MacBook’s fan. The following script helps add fine-tuning for the fan that will increase its baseline speed and ramp it up gently, so it’s not an all-or-nothing kind of setup.

$ yaourt -S mbpfan-git
# systemctl enable mbpfan.service
# systemctl start mbpfan.service

By default the service runs in verbose mode which adds tons of output to the system journal. It basically works by measuring the CPU temp, adjusting the fan speed accordingly, and then sleeping for a given number of seconds. With verbose mode on, it logs its wakeup every few seconds. That means a lot of writing to the journal. So I changed the service under /usr/lib/systemd/system/mbpfan.service so the following line reads like so:

ExecStart=/usr/sbin/mbpfan -f

Apple Keyboard

To get the fn keys working in X, we will install xbindkeys which helps bind keyboard keys to commands. We will bind keys for volume, keyboard and display brightness.

# pacman -S xbindkeys
yaourt -S xorg-xbacklight kbdlight

Create your configuration file for xbindkeys: .xbindkeysrc

# Increase volume 5% with Apple volume up
"amixer set Master playback 5%+"
    m:0x0 + c:123
    XF86AudioRaiseVolume
# Increase volume 5% with F12
"amixer set Master playback 5%+"
    m:0x0 + c:96


# Decrease volume 5% with Apple volume down
"amixer set Master playback 5%-"
    m:0x0 + c:122
    XF86AudioLowerVolume
# Decrease volume 5% with F11
"amixer set Master playback 5%-"
    m:0x0 + c:95


# Mute with Apple mute
"amixer set Master toggle"
    m:0x0 + c:121
    XF86AudioMute
# Mute with F10
"amixer set Master toggle"
    m:0x0 + c:76


# Suspend system
"systemctl suspend"
    m:0x0 + Mod4 + c:107
    Mod4 + XF86Eject


# Dim keyboard
"kbdlight down"
    m:0x0 + c:237
    XF86KbdBrightnessUp


# Brighten keyboard
"kbdlight up"
    m:0x0 + c:238
    XF86KbdBrightnessDown

Next we’ll auto-start xbindkeys when X starts, add this to your .xinitrc file:

if [ `type -P xbindkeys` ]; then
  # Load custom keyboard key bindings
  xbindkeys
fi;

Apple Trackpad

Finally¹⁰, we’ll setup a Bluetooth Apple Trackpad. To do this, we’ll install some bluetooth core utilities

# pacman -S bluez bluez-utils
# modprobe btusb  # Make sure bluetooth kernel module is loaded
# systemctl enable bluetooth.service  # Start bluetooth on reboot
# systemctl start bluetooth.service

To actually connect to the Trackpad, we’ll use the bluetoothctl interactive utility to scan-for, discover, pair and connect to the Trackpad. (If your Trackpad is already paired with another device, make sure to turn it off, then long-hold the power button until the LED flashes.)

$ bluetoothctl
> power on
> scan on
> agent on
> devices    # You should see the MAC address of your Trackpad appear if its in discoverable mode
> pair mac 28:37:37:2B:42:7A
> connect 28:37:37:2B:42:7A

Then we’ll setup some udev rules so USB Bluetooth is activated and loade when the system boots up.

# /etc/udev/rules.d/10-local.rules
# Set bluetooth power up
ACTION=="add", KERNEL=="hci0", RUN+="/usr/bin/hciconfig hci0 up"

Conclusion

Wrapping up, I have nothing to add other than what you do with your system is up to you.

References

The following articles were very useful in providing some help and inspiration.

• Installing Arch Linux on a MacBook Pro - Michael Chladek
• Introductory MacBook Arch Guide - Zanshin
• Arch Linux on MacBook Pro Retina 2014 with DM-Crypt, LVM and suspend to disk - Loïc Pefferkorn
• ArchWiki MacBook
• ArchWiki MacBookPro11,x
• Arch Linux Running on my MacBook - Phil Plückthun
• My personal fight against the modern laptop

Any changes to this article will be annotated with a footnote and explained here.

Index

• JavaScript: The Good Parts
  • Primitives
  • Execution Flow
  • Equality
  • Loops / Enumeration
  • Object Literals
  • Arrays
  • Delete
  • Global Abatement
  • Reference
  • Reflection
  • Functions
  • Prototype
  • Function Invocation Pattern
  • Constructor Invocation pattern
  • Closure
• Modules
• WTF

JavaScript: The Good Parts

A walkthrough many of the useful code examples from Douglas Crockfords’ seminal book, JavaScript: The Good Parts.

Primitives

JavaScript Reserved words:

abstract
boolean break byte
case catch char class const continue
debugger default delete do double
else enum export extends
false final finally float for function
goto
if implements import in instanceof int interface
long
native new null
package private protected public
return
short static super switch synchronized
this throw throws transient true try typeof
var volatile void
while with

Simple types

// Numbers: 1, 1.1, 1e10, 1e+10,
// Strings: "Hello", 'hello'
// Booleans: true, false
// null
// undefined
// Object

Built in values

true;
false;
null;
undefined;
NaN;
Infinity;

JavaScript represents all numbers internally as a 64-bit floating point value (like Java’s double).

// integer
1 === 1.0; // true

// fraction
(0.1 ===
  0.1(
    // true

    // exponent: value is calculated by multiplying the part before the
    // e by 10 raised to the power of the part after the e.
    1 * 10,
  ) *
    (1 * 10)) ===
  1e2; // true
1e2 === 100; // true

// Infinity represents all values greater than:
1.7976931348623157e103;
typeof Infinity === "number"; // true

Numbers, strings and booleans are object like in that they have methods, but they are immutable.

Arrays, functions, regular expressions and objects are Objects. A simple object:

// simple object literal
var an_object = {};
// simple object literal with 2 values (comma-separated)
var another_object = {
  my_value: 3.1,
  your_value: 3.2,
};
another_object.my_value == 3.1; // true

// attempts to access undefined properties will throw TypeError
another_object.fake.my_value; // Throws TypeError
// Prevent TypeError by testing that all attributes exist before accessing
(another_object.test && another_object.test.test) || "none"; // "none"

Objects are class-free, which is to say that there is no constraint on the names of its properties or their values. The object doesn’t need to be defined before it is instantiated.

JavaScript has prototype linkage, meaning an object can inherit the properties of another object. This can be used to reduce object initialization time, memory consumption and increase code reuse.

Execution Flow

Execution and flow statements.

// Conditional statements: if and switch
// Looping statements: do, for, while
// Disruptive statements: break, continue, return, throw

Falsy values. When conditional expressions evaluate to these values, the expression will result in false.

false;
null;
undefined;
(""); // empty  string
0;
NaN;

Operators and statements of equality.

// Operator precedence
. [] ()                   // Refinement and invocation
delete new typeof + - !   // Unary operators
* / %                     // Multiplication, division, remainder
+ -                       // Addition/concatenation, subtraction
>= <= > <                 // Inequality
=== !==                   // Equality
&&                        // Logical and
||                        // Logical or
? :                       // Ternary

NaN

// NaN is a number value that is the result of an operation that
// cannot produce a normal result. It is not equal to anything else,
// including itself.
NaN == NaN; // false
NaN === NaN; // false

// Use isNaN to detect if a value is NaN
isNaN(NaN); // true

Equality

Rule of thumb, the following expressions are all false using === or !== so use these operators instead.

"c" + "a" + "t" === "cat"; // true
"" == "0"; // false
0 == ""; // true, both are falsy
0 == "0"; // true
false == "false"; // false
false == "0"; // true
false == undefined; // false
false == null; // false
null == undefined; // true

Loops / Enumeration

The for in statement can loop over all properties of an object, but will include all object types (including functions) as well as all objects along the prototype chain. The for in operator enumerates the property names on an object.

var animal_names = {
    'dog': 'Piper',
    'cat': 'Snookins',
    'goat': 'Mr J'
};
for( var animal in animal_names ) {
    console.log( animal, animal_names[animal] );
}
> dog Piper
> cat Snookins
> goat Mr J

It is important to know the names can be in any order and can be any type including functions.

Object literals

Object literal - pair of curly braces surrounding zero or more name: value pairs.

var empty_object = {};
var stooge = {
  first_name: "Jerome",
  last_name: "Howard",
  nickname: "Jer",
};
var jer1 = stooge["first_name"];
console.log(jer1); // "Jerome"
var jer2 = stooge.first_name;
console.log(jer2); // "Jerome"
var jer3 = stooge.favorite && stooge.favorite.color; // return undefined
console.log(jer3); // undefined

Arrays

Arrays have a length property.

var empty = [];
var numbers = [
  "zero",
  "one",
  "two",
  "three",
  "four",
  "five",
  "six",
  "seven",
  "eight",
  "nine",
];
empty.length > 0;
numbers.length > 10;

Objects have no length property

var numbers_object = {
  0: "zero",
  1: "one",
  2: "two",
  3: "three",
  4: "four",
  5: "five",
  6: "six",
  7: "seven",
  8: "eight",
  9: "nine",
};
numbers_object.length > undefined;

var myArray = [];
myArray.length > 0;

Arbitrary array indexes can be set, although there may not be items populating the array before it.

// myArray = [];
myArray[1001] = true; // create and set value of index 1001 to true
console.log(myArray.length); // 1002 OMG!!

var arrayItemCount = 0;
for (var item in myArray) {
  arrayItemCount += 1;
}
console.log("myArray counted items: " + arrayItemCount);

To delete from an array, use Array.splice(). This function takes 3 arguments, start the index to start adding/removing items, deleteCount the number of items to remove/replace, and finally n-arguments for the addition n-items to add to the array.

var animals = ["cat", "dog", "pork", "pig", "fish"];

// remove 'pork'
a.splice(2, 1);
// removed 'pork'
a > ["cat", "dog", "pig", "fish"];

// add 'snake' before item 2.
a.splice(2, 0, "snake") > ["cat", "dog", "snake", "pig", "fish"];

viola!

Delete

The delete operator removes a property from an object. If will not remove the property if it only exists on parent objects along the prototype chain. It is useful for removing the property from a leaf object so that the property along the prototype chain is used.

Global Abatement

Global variables in general weaken the portability and resiliency of programs, but in the case they’re necessary, aim to limit them to a single global variable. This is done by attaching all properties to a single global variable.

var MYAPP = {};
MYAPP.views = {};

This will reduce namespace or scope collisions with other applications or libraries.

Reference

JavaScript passes objects around by reference, never by value.

var x = {
  fn: "Jerome",
  ln: "Howard",
  nn: "Jer",
};
var y = x.nn; // new variable value: 'Jer'
x.nn = "curly";
console.log(y); // 'curly'

Reflection

Reflection is the ability of a program to introspect is own code at runtime.

Javascript offers the typeof operator to check the type of a property, and hasOwnProperty() which return true if the property exists directly on an object, false if it does not (and possibly exists up the prototype chain.)

var a_var = "a message";
typeof a_var;
> 'string'

var an_object = {
    a_prop: true;
};
an_object.hasOwnProperty('a_prop');
> true

Functions

”The craft of programming is the factoring of a set of requirements into a set of functions and data structures.” Douglas Crockford

Functions encapsulate a set of statements. They enable code reuse, information hiding and behavior composition.

Functions in JavaScript are objects. Objects are collections of name-value pairs that have a hidden prototype link to Object.prototype. The Objects produced from object literals are linked to Object.prototype. Function objects are linked to Function.prototype– which is linked to Object.prototype. Functions are also created with two additional hidden properties: the function’s execution context (lexical scope) and the source code that implements the function.

Functions have a prototype property whos value is an object with a constructor property whos value is the function. This is distinct from the hidden link to Function.prototype. [](#ToDo: this needs to be fleshed-out in more detail)

When invoked, functions receive two additional parameters: this and arguments. The value of this depends on how the function was invoked (invoked as method, invoked as function, invoked as constructor, and invoked as apply). The value of arguments is an array-like object (in that the object has a length property) of all parameters passed to the function when invoked.

Method Invocation Pattern

When a function is created as a property of an object, it is called a method. When a method is invoked, it is bound to the object it is a property of. You can tell a function is invoked as a method when it uses a refinement; a . notation to call the function.

Functions invoked as a method can access the bound object using the this reference to access other properties and values on that object. The methods this object is late-bound at function invocation time. This makes methods that make references to this more portable. These type of objects are called public methods.

var myObj = {
  value: 2,
  increment: function (inc) {
    this.value += typeof inc === "number" ? inc : 1;
  },
};

The binding of this happens very late, at invocation time. A method can use this to retrieve values from the object, or modify the object.

Crockford introduces the method method, it is used throughout the book to define new methods on existing objects. Crockford attaches it to the Function prototype, so it is available to all objects.

Function.prototype.method = function (name, func) {
  this.prototype[name] = func;
  return this;
};

A use of the method method would be to define a new function predictLength on the String object. This function will return the predicted length a string when concatenated with the existing string.

// use Crockfords' `method` method on the `String` object.
String.method("predictLength", function (arg) {
  var len = this.length,
    argLen = arg.len,
    nextLen = len + argLen;
  var message = "Predicted length is %0 for string '%1'"
    .replace("%0", nextLen)
    .replace("%1", arg);
  console.log(message);
  return nextLen;
});

Function Invocation Pattern

When a function is not a property of an object, it is invoked directly as a function:

var sum = add(3, 4);

Any function invoked in this pattern, the this value is bound to the global object. This becomes confusing for many developers when defining objects with sub-functions that expect the value of the this value to refer to the containing object. A example of problematic code illustrates:

var brokenObj = {
  value: 2,
  double: function () {
    var helper = function () {
      // broken, the value of `this` in `this.value` refers to the global
      // Object, not `brokenObj`.
      this.value = this.value * 2; // fails, global Object.value set to NaN
    };
    helper();
  },
};

Constructor Invocation Pattern

If a function is invoked with the new prefix e.g. var x = new Status('confused'), then a new object is created and returned with a link to the value of the function’s prototype property. The value of this is bound to the new object.

var Status = function (status) {
  this.status = status;
  this.valueOf = function () {
    console.info(`status ${this.status}`);
    return this.status;
  };
};
var happy = new Status("happy");
typeof Status === "function"; // true
typeof happy === "object"; // true
happy.valueOf() === "happy"; // true
("status happy");
true;

Apply Invocation Pattern

The apply method is available on every function. In the constructor invocation pattern above, the valueOf method was bound to the local object. Instead, the the apply method allows the developer supply an arbitrary value for this, in effect allowing the developer to apply the method to any object rather than to the object it is currently bound too.

var DoMath = function (x, y) {
  this.x = x;
  this.y = y;
  this.sum = function () {
    return this.x + this.y;
  };
};
var oneTwo = new DoMath(1, 2);
oneTwo.sum() === 3; // true

// now we can force sum() to reference an object that we pass to the apply() method
var threeFour = {
  x: 3,
  y: 4,
};
oneTwo.sum.apply(threeFour, null);
7;

Prototype

Every object is linked to a prototype from which it inherits properties. All objects created with object literal syntax are linked to Object.prototype. When creating new objects, we can select which object it inherits from. We can use a method named Object.create to implement this behavior.

Object.create

if (typeof Object.create !== "function") {
  Object.create = function (o) {
    var F = function () {};
    F.prototype = o;
    return new F();
  };
}

The prototype link is used only in retrieval. If we try to retrieve a property value from an object, it will either return the value on the object itself, or look up to its prototype, and so on, until the lookup finds the property or undefined is returned if nowhere along the prototype chain nor Object.prototype has the property. This is called prototype retrieval delegation.

If properties are added to a prototype, all children of that prototype will immediately be available.

To determine if an object has a property directly, without invoking the prototype chain, use hasOwnProperty.

var ob = {};
ob.hasOwnProperty('newProperty');
> false
eb.newProperty = "a value";
ob.hasOwnProperty('newProperty');
> true

Prototypal inheritance

// differential inheritance
var Mammal = {
  name: "A Mammal",
  says: "Noise",
  get_name: function () {
    return this.name;
  },
  speak: function () {
    return this.says || "";
  },
};
var myCat = Object.create(Mammal);
myCat.name = "Henrietta";
myCat.says = "meow";
myCat.purr = function (n) {
  var i,
    s = "";
  for (i = 0; i < n; i += 1) {
    if (s) {
      s += "-";
    }
    s += "r";
  }
  return s;
};
myCat.get_name = function () {
  return this.speak() + " " + this.name + " " + this.speak();
};

For-in loops

var stooge_2 = Object.create(stooge);
stooge_2.first_name = "Harry";
stooge_2.last_name = "Moses";
stooge_2.nickname = "John";
for (var name in stooge_2) {
  console.log(stooge_2[name]);
}

delete stooge_2.nickname;
console.log(stooge_2.nickname); // 'curly'

// Method invocation pattern
console.log("===== Method invocation pattern ====");
var myObject = {
  value: 0,
  increment: function (inc) {
    this.value += typeof inc === "number" ? inc : 1;
  },
};

myObject.increment();
console.log(myObject.value); // 1
myObject.increment(2);
console.log(myObject.value); // 3

Function invocation pattern

myObject.double = function () {
  var that = this; // global this mis-assignment fix
  var helper = function () {
    that.value = that.value * 2;
  };
  helper(); // invoke helper as a function
};
// invoke double as a method
myObject.double();
myObject.value > 6;

Constructor Invocation pattern

var Quo = function (string) {
  this.status = string;
};
Quo.prototype.get_status = function () {
  return this.status;
};
var myQuo = new Quo("confused");
console.log(myQuo.get_status()); // 'confused'

// Apply Invocation pattern
console.log("===== Apply Invocation pattern =====");
var statusObject = {
  status: "A-OK",
};
// invoke the get_status method from Quo on our statusObject
var status = Quo.prototype.get_status.apply(statusObject);
console.log(status); // 'A-OK'

// Function Parameters
console.log("===== Function Parameters =====");
var sum = function () {
  var i,
    sum = 0;
  for (i = 0; i < arguments.length; i += 1) {
    sum += arguments[i];
  }
  return sum;
};
console.log(sum(4, 8, 15, 16, 23, 42)); // 108

// Exceptions
var add = function (a, b) {
  if (typeof a !== "number" || typeof b !== "number") {
    throw {
      // throw an object with a name and message property
      name: "TypeError",
      message: "add need numbers",
    };
  }
  debugger;
  return a + b;
};
var try_it = function () {
  try {
    add("seven");
  } catch (e) {
    console.log("name: " + e.name + " msg: " + e.message);
  }
};
try_it();

Object augmentation

Function.prototype.method = function (name, func) {
  // add arbitrary functions with name name to our Object.
  this.prototype[name] = func;
  return this;
};
Number.method("integer", function () {
  return Math[this < 0 ? "ceil" : "floor"](this);
});
console.log((-10 / 3).integer()); // -3

Lexical Scope

Lexical scope is the availability of variables to an executing block of code. JavaScript lexical scope is determined by resolving objects in the immediate lexical scope, then moving up the chain until it reaches the variables defined in the global scope.

var foo = function() {
    var a = 3,
        b = 5;
    var bar = function() {
        var b = 7,  // define local b
            c = 11; // define local c
        // a = 3, b = 7, c = 11
        console.log('a', a, 'b', b, 'c', c);
        a += b + c; // 3 + 7 + 11
        // a = 21, b = 7, c = 11
        console.log('a', a, 'b', b, 'c', c);
    };
    // now: a = 3, b = 5, c = undefined
    console.log('a', a, 'b', b);
    bar();
    // a = 21, b = 5
    console.log('a', a, 'b', b);
};
foo();
> a 3 b 5
> a 3 b 7 c 11
> a 21 b 7 c 11
> a 21 b 5

Closure

A closure is a reference to a function as well as the runtime environment of the function. This includes any variables within the lexical scope of the function.

// addN returns a function that adds its argument.
var addN = function (x) {
  // x and the function below are in the same lexical scope
  return function (y) {
    return x + y;
  };
};

// add3: Return a function that adds 3 to its argument
var add3 = addN(3);
// add5: Return a function that adds 3 to its argument
var add5 = addN(5);
var resultOfAdd3 = add3(4); // 7
var resultOfAdd5 = add5(4); // 8
resultOfAdd3 > 7;
resultOfAdd5 > 8;

Additional examples of closures in JavaScript:

var closureObject = (function() {
    var y = 0; // this variables lifetime is always available to
               // functions in the same lexical scope.
    return {
        increment: function(x) {
            y += typeof incrementBy === 'number' ?
                x :
                1;
        },
        getValue: function() {
            return y;
        }
    }
}());
// Assign the result of returning a self-executing function to
// closureObject.
// The function returns an object with 2 methods, each
// maintains access to the `y` variable.
closureObject.increment(3);
closureObject.getValue();
> 3

// Demonstrates the preservation of the lexical scope of execution.
var quo = function(status) {
    return {
        get_status: function() {
            return status;
        }
    };
};
var newQuo = quo('amazed!');
// newQuo still has access to the status parameter even after
// execution has ended and the lifetime of its lexical scope has ended.
// This is possible because the function has access to the lexical
// scope in which it was created. This is called a closure.
newQuo.get_status();
> 'amazed!'

Closure can be useful for things such as callbacks used in animation.

var fade = function (node) {
  var level = 1;
  var step = function () {
    var hex = level.toString(16);
    var colorValue = "#ffff" + hex + hex;
    node.style.backgroundColor = colorValue;
    console.log("level " + level + " backgroundColor: " + colorValue);
    // step() maintains access to 'level' variable outside its execution scope via Closure.
    if (level < 15) {
      level += 1;
      setTimeout(step, 100);
    }
  };
  setTimeout(step, 100);
};
fade(document.body);

Using closure and setTimout to increment a local variable.

for (var i = 0; i <= 10; i++) {
  (function (i) {
    setTimeout(function () {
      console.log(i);
    }, 1000 * i);
  })(i);
}

Modules

A module is a function or object that presents an interface but that hides its state and implementation. —Douglas Crockford

Modules exist because developers want to write code in discrete JavaScript files, while [web] deployment wants optimized code in as few HTTP requests as possible. These interests are add odds with one another.

Modules aim to encapsulate a piece of code into a useful unit, and register its capability/export a value for the module.

The general pattern of a module is a function that defines private variables and functions; creates privileged function which, through closure, will have access to the private variables an functions; and that returns the privileged functions or stores them in an accessible place. —Douglas Crockford

Douglas then defines a simple module using the function-closure pattern, which returns an anonymous function to a global variable where the module will be referenced.

// SayModule: A function that simply console.log's its argument.
var SayModule = ( function() {
    var message = 'You said: {0}';
    return function(arg) {
        console.log( message.replace( '{0}', arg ) );
        return arg;
    }
}() );

var whatSaid = SayModule("hello world!");
> You said: hello world!
> 'hello world!'
whatSaid
> 'hello world!'

With many of these, module scripts are strung together in the DOM with namespaces being described by a single global object where it’s still possible to incur naming collisions in the architecture.

CommonJS is a set of conventions used to define how JavaScript modules are used and packaged. It defines the Module specification. This describes how code can fetch and include external modules as a dependency using the require("<name>") function. It also defines how authors export their code either by reassigning a module.exports object, or adding properties to an exports variable. CommonJS is most often used for synchronous loading of dependencies on server-side systems such as Node.js.

AMD (Asynchronous Module Definition) is another specification for modules. RequireJS is probably the most popular implementation of AMD. RequireJS was designed to asynchronously load modules in a browser context. RequireJS implements AMD, which is designed to suit the async nature of a browser. RequireJS implements a define() function that allows the module to declare its dependencies before being loaded. RequireJS can easily be wrapped by CommonJS such that CommonJS modules can easily be included using RequireJS.

Browserify is a CommonJS Module implementation that can run in the browser. Browserify introspects JavaScript use of the CommonJS require() function and builds a “bundle” that can be served in a single HTML <script> tag.

webpack is a module bundler that aims to understand the needs of the developer as well as solve the problems of delivering code modules efficiently in the browser. webpack uses an opinionated configuration-over-code approach (similar to Grunt, but with more opinions) vs Gulp, which requires much more manual wiring.

WTF

0.1 + 0.2 // 0.3? NOPE!
> 0.30000000000000004

x = 1.0000000000000001
x === 1
true // Huh?

y = {}
> y[[]] = 1
> Object.keys(y)
[ '' ] // HuH?

Logging

Logs are time-ordered streams: there is no beginning or end, but rather an ongoing, collated collection of events which we may wish to view in realtime as they happen. Unix provides some excellent tools for handling streams. There are two default output streams, stdout and stderr, available automatically to all programs.

A program which uses stdout is equipped to log in a variety of ways without adding any weight to its codebase or configuration format.

Treating your logs as streams is a form of future-proofing for your application. Choosing to use stdout over custom-implementing a specific logging solution allows your application to change logging mechanisms with 0-code changes. It allows you to be the most agnostic as you haven’t needed to make any decisions or implementations other than adopting a long-standard convention.

If you run them in the foreground, as is typical of development mode, you see the output right in your terminal. This is exactly what you want. If you run in production mode, you can redirect the output to a file, to syslog, to both, or to any other logging system that can accept an input stream.

Logging on any reasonably large distributed system will generally end up using the syslog protocol to send logs from many components to a single location. Programs that treat logs as files are now on the wrong path: if they wish to log to syslog, each program needs to implement syslog internally - and provide yet more logging configuration options to set the various syslog fields.

Best Practices

1. An app shouldn’t implement a custom logging solution. Simply write to stdout and stderr.
2. Don’t write to a log file, and don’t expect log files to be managed. This then requires log rotation and log file maintenance.

During local development, the developer will view this stream in the foreground of their terminal to observe app behavior. During production the runtime environment will read stdout and stderr from the app, the streams will be captured by the execution environment, collated together with all other streams from the app, and routed to one or more final destinations for viewing and long-term archival. These archival destinations are not visible to or configurable by the app, and instead are completely managed by the execution environment. Furthermore, the app needn’t implement any logging solution.

The event stream for an app can be rerouted to a file (if needed), or watched in a terminal. Most significantly, the stream can be sent to a log indexing and analysis system such as Splunk. These systems allow for great power and flexibility for introspecting an app’s behavior over time, including:

• Finding specific events in the past.
• Large-scale graphing of trends (such as requests per minute).
• Active alerting according to user-defined heuristics

In Practice

Already using Amazon AWS? Checkout CloudWatch, the additional advantages here are that you could have a central source of truth for all monitoring needs because such metrics as CPU, disk I/O, and network for your container instances are already available on CloudWatch.

If using Docker, Docker 1.9 announced a logging driver for CloudWatch. Use these options to enable the awslogs Amazon AWS CloudWatch logging driver:

--log-opt awslogs-region=<aws_region>
--log-opt awslogs-group=<log_group_name>
--log-opt awslogs-stream=<log_stream_name>

Provide AWS credentials to the Docker daemon to use the awslogs logging driver. You can provide these credentials with the AWS_ACCESS_KEY_ID , AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables.

Credentials must have a policy applied that allows the logs:CreateLogStream and logs:PutLogEvents actions, as shown in the following example.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Use containers to move logs from one container into another service using a Docker logging driver. Docker allows configuration of container log driver:

container_name:
    log_driver: syslog
      log_opt:
        syslog-tag: nginxproxy_nginx
        syslog-address: udp://MY_DOCKER_HOST

Using Splunk? Use containers to run a Splunk forwarder

splunkforwarder:
  image: outcoldman/splunk:6.2-forwarder
  environment:
    - SPLUNK_FORWARD_SERVER=YOUR_DOCKER_HOSTNAME:9997
  volumes_from:
    - vforwarder
  ports:
    - 514:1514/udp
  restart: always

A Retrospective — Responsive Design

The mark of a master woodworker is his/her ability to work with the grain. The grain gives the wood its natural identity and subtly tells the craftsman the best way to work it.

Work with the grain and save effort. Work with the grain and maximize flexibility. Work with the grain and maintain strength.

Work against the grain at the expense of additional effort on the part of the craftsman as well as reduced flexibility and strength in the wood.

The same can be said for crafting products on the web. The raw materials are HTTP requests, HTML, text, CSS, Javascript, images, and motion.

To go with the flow of the web-grain is to deliver content quickly, with vertically-variable content-flow. Reduce HTTP requests and add complex functionality using progressive enhancement.

The tools that we design should be user-centric and our metric of success is when our user puts our tools down. In the end, we want our users to be engaged with the people and world around them, not fumbling with a tool. The tools are a means, not an end.

s/Techn/Method/ology

Server-side web frameworks like Django, Ruby on Rails, Express all have excellent technologies to define explicit dependencies (e.g., pip with requirements.txt, gem with Gemfile, and npm with package.json.) Configuration management tools like Chef, Puppet, Ansible and Docker are making the server-environment both easier to define implicit dependencies and help automate the task of creating homogeneous development and production environments. Together, these tools can build a reproducible execution environment where the code can be more fully tested in a space with less far less entropy than the web environment.

The execution environment of the web is extremely heterogeneous: it operates on a fractured multitude of devices within a dozen of browser of various capability. Bandwidth, CPU speed & memory, screen size & pixel density are just a few of the multitude of variables web developers need to manage. This diversity of environment makes for some very complex problem-solving, but that very heterogeneity is the grain of the web, it’s what gives it resilience and staying-power; it’s the power of decentralization at work.

Responsive design was born out of the complexity and diversity of the web. It is not a technology but a methodology used to cope with the diversity of environments a web product needs to operate within.

We use this methodology to engage in conversation that is optimized for each of our users. We do this by taking device-capability, user attention-levels and context into account; and designing specifically for these variables. This is opposed to the familiar approach where we send a canned “one-for-all” solution to everyone (and yet no-one in particular.)

“It’s important to note the different between support and optimization. Optimizing for every web-enabled device on earth is impossible, so eventually you need to make the strategic (read: business) decision of what target devices and browsers for which to optimize.” —Brad Frost

If we bifurcate our web experiences into arbitrary “web” and “mobile” buckets, we will then be forced to decide which bucket any new device best fits in. Can we really continue to commit to supporting each new device (as the market continues to fracture) with a bespoke experience? Doubtful.

Instead we should shift our thinking: How can our designs best-adapt and flow into the context and environment in which our users operate; in order to best serve the goals of the user?

“The control which designers know in the print medium, and often desire in the web medium, is simply a function of the limitation of the printed page. We should embrace the fact that the web doesn’t have the same constraints, and design for this flexibility. But first, we must ‘accept the ebb and flow of things’.” -John Allsopp, “A Dao of Web Design”

Up front we should make an important distinction between “responsive design” and “web design”; namely that experience design is about purpose, while web design is about graphic aesthetics. (Most often, “web design” is just a reference to a picture, a design comp.)

While the design comp can refer to the aesthetics of the product, it doesn’t communicate its essences of motion, interaction, performance or scalability. Often times this communication exists completely outside the comp (in a discussion or meeting), or alongside it. The evidence of this gap in communication is the reliance on design-document annotations using the following words:

• “Fades”
• “Jumps”
• “Flips”
• “Shrinks”
• “Collapses”

That said, when considering whether or not to implement experience design into business strategy, a major trade-off to keep in mind is that marketing and graphic design stakeholders will be relinquishing control to other stakeholders. An experience-oriented approach can make the marketing team out feel like they’ve been taken out of the driver’s seat. The graphic design team needs to be ready for a challenge, amicable to iteration and prototyping as well as forgiving about variability in the final outcome.

In order to advocate for the grain of the web, our job must become political and inclusive. We need to use sensitivity, language and organizational politics to get all stakeholders to reach consensus.

“Like celebrities on the cover of a magazine, ‘pixel-perfect’ execution on the web is only ever achieved in Photoshop.”

What “Responsive design” Is

• A process
• A relinquishing of the false sense of comfort we get when designing for a “design comp”
• An experience-first / MVP friendly approach to building dynamic, consistent and importantly a complete experience that adapts to users context
• A conduit for the flow of of communication, and much much like water, it flows into different browsers & devices, flows into robot search engine crawlers, flows into RSS feeds, flows into meta-data (hReview) parsers, flows into printers, flows into read-later tools like Web Clippings, Pocket, Instapaper and FlipBoard

Responsive Design as a catalyst for organizational change

Responsive design is more of an organizational challenge than a technical challenge.

The challenge for teams structured traditionally into Business, Product, UX, Design, Development, QA is that they operate in a waterfall process, with little-to-no feedback cycle and a lack of cross-disiplinary team member involvement. Accountability to the result moves top-down and never bottom-up. Product has little push-back on business, UX has little weight to affect product decision, and then down the waterfall, design has little affect on the UX decisions. Once in development, the dev-team has little affect backwards up the chain. This always guarantees last-minute compromise and needless wasted effort.

As the workflow gets closer to Launch, changes to the design become more and more expensive. This creates an expensive, rigid and compromised final product. Most importantly, the end-users aren’t impressed and team members are annoyed with all the compromises made. Nobody is happy.

Using a cross-disciplinary, phased collaborative approach, we benefit from deliverables that can validate our decisions much earlier in the process, and importantly- those decisions are less complex (read: less expensive) to change.

This process is called the Spiral Model Process and it goes hand-in-hand with Responsive Design Methodology. All team members are involved in the development of the product early and often. Team members feel like they have more input on a wider aspect of the product, and this results in higher levels of personal responsibility and acts as an intrinsic motivator for all team members.

Just remember, Responsive Design isn’t a set of CSS features, a technology stack or a set of tools. Its when an organization decides to collaborate around a process; with accountability to the user first, to each other second, and to solve the technology last, once all important decisions have been made.

More reading on “Responsive Design” and organizational change

• Upstatement & Filament Group discusses their approach to the redesign of the Boston Globe
• Matt Andrews on the responsive redesign at the Guardian UK

This article aims to examine the benefits of Docker within the context of a software company with multiple developers working on multiple projects, having to manage these projects deployed in a cloud production environment. In this context the benefits of Docker become more readily apparent.

With the rise of cloud computing, the number of systems that need to be maintained has exploded. Manual provisioning of an increasingly large number of systems becomes impossible for a small team, given platforms like Amazon EC2 provide auto-scaling when additional load is detected.

Tools like Ansible, Chef, Puppet and Salt are great solutions towards achieving automated provisioning of virtual machines. The industry has responded by quickly adopting these tools, but even more agility and performance can be achieved by using Docker containers.

Combining Docker with a deployment tool like AWS Elastic Beanstalk can provide even greater efficiencies for developing and deploying cloud applications.

Dockdj

This article will be using Dockdj to illustrate using Docker and Elastic Beanstalk in the context of a real-world web project. Dockdj is a recipe for building 12-factor Python / Django web apps with multi-container Docker and deploying to Amazon AWS using Elastic Beanstalk. Dockdj is available on GitHub.

Manual Provisioning

The naive approach is manual provisioning: the developer installs Apache and associated system libraries directly on the local development machine, configure it according to the WordPress documentation.

These manual steps will need to be repeated for every additional member of the development team, and again for the production web server. When provisioned software is updated or configurations change. All members of the development team and the production systems need to be updated accordingly. Larger teams inevitably begin experiencing the “works on my machine” problems between developers when some developers haven’t updated their configurations to match coworkers who have.

Manual provisioning quickly becomes a frequent and resource-intensive process, with the side-effect of prolonging the deployment of important vendor bug fixes and security patches across both development and productions systems.

Additionally, when increased traffic hits productions systems, new systems need to be allocated and scaled horizontally to support the new traffic. All the provisioning needs to repeated. This system doesn’t scale as more production servers are added to serve additional traffic load.

Even worse is when differences between developer-systems and production-systems result in hard-to-reproduce bugs once the app is deployed from development into production.

Automated Provisioning

An improvement over manual provisioning is automated provisioning using a configuration management tool like Ansible, Chef, Puppet, Salt, etc. These tools have been developed to address the problems of provisioning at large scale.

The aim of these tools are:

1. Initialize and start virtual machines
2. Automate the provisioning process in a repeatable way
3. Manage changes to provisioning in a version control system
4. Establishing and maintaining consistency of system dependencies and configuration throughout an applications life

Configuration management tools are wonderful for automated provisioning, but in practice¹ they tend to split management of the stack-app into two parts:

1. the software Stack

• Operating system
• System libraries
• Provisioned software
• Configuration

1. the Application

• Source code & binaries
• Dependencies
• Runtime environment

The result is that the stack (#1) is initially allocated and provisioned using one of the configuration management tools. The application (#2) is then deployed on the stack— resulting in a running application. When subsequent application versions (#2) are released and deployed, they are deployed onto the (unchanged) stack. The problem with this model is that the stack and the application are managed independently. Changes to the stack are managed as a unit separate from changes to the application. No data is recorded that describes the compatibility of the integrated whole.

This results in increased complexity during rollbacks or simultaneous updates to both stack and application. More importantly version numbers of the application are not tied to versions of the stack.

Under this model, the stack version and application version aren’t coupled— which increases the likelihood of integration failures.

An example will illustrate where this model will fail:

Our production web server is provisioned with Apache 3.3.0 and the application (WordPress) was at version 0.7.0 last week, and have just released version 0.8.0 this past week.

Apache announces a security vulnerability fix at version 3.3.1. Under the (typical) automated provisioning model, the configuration management tool would be updated to provision the new version of Apache. The tool runs against all production server systems. Here the application doesn’t change, it simply rides on top of the Apache stack without change. No problems occur with the rollout of the new Apache release.

Next the application updates and releases a new version for deployment- 0.9.0. The deploy process runs, and for some reason the application fails, it isn’t compatible with version 3.3.1 of Apache.

The decision is made to rollback the application to 0.8.0, which runs successfully with Apache 3.3.1. The system is working again.

A critical security vulnerability is discovered in application 0.8.0 and the decision is made to roll application back to version 0.7.0. (Keep in mind the previous app version 0.7.0 was running Apache 3.3.0, and the stack is currently 3.3.1).

The application fails— because 0.7.0 was never integration tested against Apache 3.3.1. What do you do?

In this example the devops team failed to remember to rollback Apache, simply because the integrated dependencies were not internally coupled. The compatible coupling existed only as institutional knowledge outside the scope of the configuration management system, as Stack and Application were managed separately.

Docker for Configuration Management

One major advantage of Docker is that it does not necessitate running a unique VM² for every project a team works on. If developers work on multiple projects, each with its own customized VM, switching between projects becomes a time-consuming context shift for developers.

Docker containers run directly on the Linux operating system and yet each container is isolated. This eliminates the slowness of booting and the overhead of a VM. Docker containers start up as quickly as running a normal process, and eliminate VM “booting” for every Docker project the developer works on. Deploying changes to the environment for every developer working on the project is as easy as publishing a new Docker image. Next time a developer starts the container, he/she will get the new image.

Another advantage of Docker over an automated configuration management tool is that it does not bifurcate the stack and the application into independent segments.

A stack using Docker containers has the same benefit of configuration management, but can couple the stack and the application into a single managed component. The application is deployed along with its stack— and the complete stack-app component is deployed together as a single Docker image or a bundle of Docker images that have already been integration tested at least on a developers machine.

As opposed the the “automated provisioning” model, the Docker model of the stack-app looks more like this:

1. Set of Docker images

• Operating system (the software Stack)
• Provisioned software (the software Stack)
• Configuration (the software Stack)
• System libraries (the software Stack)
• Source code & binaries (Application)
• Dependencies (Application)
• Runtime environment (Application)

With every deployment, the entire stack-app (1) will be deployed. Docker uses hashes (like Git) to minimize the amount of data that will be downloaded for any update. This means only the differences are downloaded rather than the entire stack.

The greatest advantage of using Docker is that developers can run the application in the very same environment as production. According to Twelve-Factor Methodology this is called achieving “Dev/Prod Parity”. This is a huge benefit in that it eliminates an entire class of bugs that result from differences between developers-and-developers as well as bugs that result from differences between developers-and-production.

Elastic Beanstalk for Deployment

If you know Heroku, than Amazons’ Elastic Beanstalk will be extremely familiar. EB borrows many ideas from Heroku, but the killer feature is its’ ability to dynamically run, deploy and scale Docker containers on a cluster of servers. It handles hardware allocation, network configuration, load balancing, auto-scaling, health monitoring and rolling deployments.

EB doesn’t do everything, but it’s good enough to adopt early and use until your team understands its deployment use-cases more clearly and understands limitations of EB and its trade-offs.

Django specific structure

The core structure of the Docker / Elastic Beanstalk app can be explained by describing the directory structure.

These comments describe the application-specific file structure:

.dockerignore
.ebextensions/
  01_envvars.config
.ebignore
.elasticbeanstalk/
Dockerrun.aws.json
.gitignore               # Describes which files git ignores
.bowerrc                 # Configures where web frontend dependencies live
.csslintrc.json          # Describes CSS syntax rules
.jshintrc                # Describes JavaScript syntax rules
bower.json               # Describes web frontend dependencies
gulpfile.js              # Describes app build and dev tasks
package.json             # Describes NPM dependencies
app/                     # Our python app
  apps/*                 # python app modules
  project/*              # App-specific settings
  dist/*                 # App static assets (served via Nginx)
bin/*
docker/
  django/
    dev/
      docker-compose.yml
      Dockerfile
    prod/
      docker-compose.yml
      Dockerfile
      gunicorn.conf.py   # Settings for production app-server
    start.sh             # Script to start app-server
  nginx/*                # Nginx config files
environments/            # Environment-specific settings
  dev/                   # Development-only environment settings
    .env                 # Actual environment vars (Excluded from git)
    .env.example         # Example environment vars
    Procfile             # Configures how Honcho starts app-servers
    requirements.txt     # Describes dev Python dependencies
  prod/                  # Development-only environment settings
    .env                 # Actual environment vars (Excluded from git)
    .env.example         # Example environment vars
    Procfile             # Configures how Honcho starts app-servers
    requirements.txt     # Describes prod Python dependencies

Some of these directories and files are described in more depth below:

.gitignore

environments/*/.env
node_modules
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml
# Built testing and static asset artifacts
app/dist

Files matching the name environments/*/.env contain sensitive information (usernames, passwords, etc) about per-deployment environments that shouldn’t be included in version control.

The node_modules directory and will be created when the developer installs NPM packages. These are dependencies and should not be committed into the source code repository.

The .elasticbeanstalk/* directory is excluded from Git because it contains files that are generated by EB command-line during environment creation and version deployment that shouldn’t be written to the repository. It also contains temporary configuration files (written by the EB cli).

Both !.elasticbeanstalk/*.cfg.yml and !.elasticbeanstalk/*.global.yml entries use the “NOT” operator to re- include themselves into the repo. These files can be useful to have in version control, as they contain useful environment configuration settings.

bower.json

Bower is a web frontend package management system. The application declares its frontend dependencies in this file.

During docker image creation, these dependencies are installed.

gulpfile.js

Gulp.js is a task runner for Node.js. gulpfile.js defines common tasks and utilities related to this application:

• Running code Syntax checking & automated testing
• SASS and CSS compilation and minification
• Frontend asset building

package.json

NPM is a package management system for Node.js applications. package.json The application declares its Node.js dependencies in this file.

app/

The app/ directory contains all source code related to the Django python web application.

docker/django/prod/gunicorn.conf.py

This project uses two application servers, runserver_plus during development and gunicorn in production.

These are settings related to the Gunicorn application server. In production, a more performant application is used, requiring this configuration file.

docker/nginx/sites-enabled.conf

On production systems, where nginx acts as a reverse-proxy for the Gunicorn web application, we use Docker links to connect the two containers together. This configuration is best for reducing latency. Inside our nginx config file, we can use a named entry for the proxy_pass value to reference our Django application server running in another container on port 8080.

# ...
location / {
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_redirect   off;
  proxy_pass       http://django:8080;
  # ...

docker/django/start.sh

cd /var/app
# ...
honcho --procfile "environments/$ENV_NAME/Procfile" \
       --env "environments/$ENV_NAME/.env" start

start.sh is used during both development and production as a single task to bootstrap the application server. It uses the honcho task runner to start the server according to a set of tasks in a Procfile for development and another for production.

The --env parameter is used to pass environment variables sourced from the environments subdirectory. At runtime, the $ENV_NAME variable will be set: dev for development and prod for production. This way a separate Procfile and separate set of environment variables are available to configure the modes independently.

environments/dev/Procfile

# The webserver: Python
webserver: cd app && ./manage.py runserver_plus 0.0.0.0:8080
# The CDN assets emulation server
cdnserver: cd /var/app/app/dist && python -m http.server 8010

During development, the Django runserver_plus application server interprets Python, while a simple HTTP server serves assets (images, CSS, etc).

environments/prod/Procfile

webserver: cd app && gunicorn \
  -c /etc/gunicorn/gunicorn.conf.py project.wsgi:application

In production, we use Gunicorn to serve the python application, so the only task run is the gunicorn app server. Static assets aren’t handled here because Nginx will be reverse-proxying the application and also serving static assets.

This decision was made because Nginx is optimized to serve static assets and it will reduce load on the application server.

environments/[dev | prod]/.env.example

Both environments/dev/.env.example and environments/prod/.env.example are committed into git to provide hints to developers that they should create a .env file in the same directory as a place to store environment variables that are passed into the application.

The .env file is excluded from the git repository as it contains sensitive usernames passwords and cryptographic information.

environments/[dev | prod]/requirements.txt

The python utility pip is a package management utility. It uses a file named requirements.txt to install all package dependencies. The development environment adds some useful debugging utilities that shouldn’t be included on production systems, so production has its own file.

During docker image creation, these dependencies are installed.

Docker specific structure

.dockerignore            # Describes which files Docker ignores
.ebextensions/
  01_envvars.config
.ebignore
.elasticbeanstalk/
Dockerrun.aws.json
.gitignore
.bowerrc
.csslintrc.json
.jshintrc
bower.json
gulpfile.js
package.json
app/
  apps/*
  project/*
  dist/*
bin/*                    # Scripts for Docker, build and deployment
docker/                  # Configuration files required by docker
  django/                # Django related settings
    dev/                 # Development-only settings
      docker-compose.yml # Orchestrates dev containers
      Dockerfile         # Builds dev Docker image
    prod/                # Production-only settings
      docker-compose.yml # Orchestrates prod containers
      Dockerfile         # Builds prod Docker image
      gunicorn.conf.py
    start.sh
  nginx/*
environments/
  dev/
    .env
    .env.example
    Procfile
    requirements.txt
  prod/
    .env
    .env.example
    Procfile
    requirements.txt

.dockerignore

The .dockerignore file specifies a list of patterns to exclude from the build context during creation of the Docker image. These files are not required by the execution of the container, and should be removed to reduce the size of the final image.

.coverage
.ebextensions/*
.elasticbeanstalk/*
.ebignore
.dockerignore
.git
.gitignore
.DS_Store
node_modules
app/dist
docs
htmlcov
README.md
ghostdriver.log

Some notable entries are described below:

The node_modules directory and will be created when the developer installs NPM packages. If the developer is using on OS X, packages compiled on OS X will not work when the container is running in the Linux VM, so this entry ensures that node modules are installed on the host OS independently from the container OS.

.ebextensions/*, .elasticbeanstalk/* and .ebignore are required by the Elastic Beanstalk deploy process, and are outside the scope of execution of the Docker container, and are not required.

The app/dist directory contains frontend assets served by both Django and Nginx, and are required by both Django and Nginx containers. Because Amazon ECS cannot (currently) directly mount a single volume from one container into another container, we need these files to be deployed directly on the host OS. Docker can mount the directory on both Django and Nginx containers as a shared volume. This directory will be deployed by EB, and is thus excluded from Docker.

bin/image

bin/image is a shell script that wraps common Docker commands used to create Docker images. There are three major subcommands: build, destroy and update all take a single argument, the name of the environment subdirectory of the environments/ directory. This will spawn Docker and build, delete or rebuild the image as specified by the Dockerfile in the directory corresponding to the final argument.

bin/stevedore

bin/stevedore is a shell script that wraps common Docker commands used to start and stop Docker containers. There are many subcommands, but the most useful are: start, stop and either of the two build commands. All subcommands take take a single argument, the name of the environment subdirectory of the environments/ directory. This will spawn Docker and start, stop or run the corresponding build process.

docker/django/dev/docker-compose.yml

This file provides configuration for Docker to orchestrate the management of the development Docker container for the local dev environment.

django:
  build: ../../..
  dockerfile: docker/django/dev/Dockerfile
  env_file: ../../../environments/dev/.env
  volumes:
    - "../../../app/apps:/var/app/app/apps"
    - "../../../app/dist:/var/app/app/dist"
    - "../../../app/project:/var/app/app/project"
    - "../../../app/manage.py:/var/app/app/manage.py"
    - "../../../environments:/var/app/environments"
    - "../../../gulpfile.js:/var/app/gulpfile.js"
  ports:
    - "80:8080"
    - "8010:8010"

It defines one container “django”, specifying a path to the build-context as well as a path to load the Dockerfile. env_file specifies the path the a file containing all environment variables. A set of volumes to share from the host OS to the container are listed in volumes. Finally ports tells Docker which ports on the host to map to the container.

docker/django/dev/Dockerfile

The Dockerfile is a set of instructions for Docker to execute in order to produce a Docker image— a file used to create a Docker container running your application code.

# ...
# Install apt, Python then NodeJS dependencies.
RUN             apt-get update && \
                curl -sL https://deb.nodesource.com/setup_0.12 | bash - && \
                apt-get install -y nodejs && \
                pip install --upgrade pip && \
                pip install -r \
                    environments/dev/requirements.txt && \
                npm update && \
                npm install -g gulp && \
                npm install && \
                gulp
# Add our initialization script to the image and run it upon startup.
ADD             docker/django/start.sh /
CMD             ["/start.sh"]

In the development Dockerfile, pip and npm commands install the necessary dependencies from the environments/dev folder. Finally start.sh is called to start the Django application server.

docker/django/prod/docker-compose.yml

This file provides configuration for Docker to orchestrate the management of the production Docker containers. This configuration can be used for testing locally prior to deployment to Amazon AWS.

django:
  build: ../../..
  dockerfile: docker/django/prod/Dockerfile
  env_file: ../../../environments/prod/.env
  volumes:
    - "../../../docker/django/prod/gunicorn.conf.py:/etc/gunicorn/gunicorn.conf.py:ro"
    - "/var/app/app/dist"

nginx:
  image: nginx
  links:
    - django
  volumes:
    - "../../../docker/nginx/nginx.conf:/etc/nginx/nginx.conf:ro"
    - "../../../docker/nginx/sites-enabled.conf:/etc/nginx/conf.d/default.conf:ro"
  volumes_from:
    - django
  ports:
    - "80:80"

It defines two containers “django” and “nginx”. “Django” is configured quite similar to the development setup, but doesn’t map as many volumes from the host OS to the container. In this configuration “django” doesn’t need to expose a port externally, as the “nginx” container will expose port 80 externally.

Importantly, the “nginx” container uses links to connect the “django” container to “nginx”. This way nginx config files can refer to “django” as-if it was another host on the same network with the name “nginx”.

Finally, the “nginx” container will mount all volumes from the “django” container with the volumes_from directive, and expose port 80 to the host OS.

docker/django/prod/Dockerfile

The major difference between the development Dockerfile and the production version is:

• It exposes port 8080 for other containers
• It runs gulp build during image creation

This Dockerfile also installs production-only Python pip dependencies in requirements.txt.

# Install apt, Python then NodeJS dependencies.
RUN             apt-get update && \
                curl -sL https://deb.nodesource.com/setup_0.12 | bash - && \
                apt-get install -y nodejs && \
                pip install --upgrade pip && \
                pip install -r \
                    environments/prod/requirements.txt && \
                npm update && \
                npm install -g gulp && \
                npm install && \
                gulp build
# Exposes port 8080
EXPOSE          8080

When Docker runs the image build, it runs gulp build, which runs code-quality, unit tests and produces production-ready web frontend assets. This allows for testing prior to deployment, and gives the team an opportunity to fix errors before they go into the wild.

Elastic Beanstalk specific structure

Finally, these comments describe files related to Amazons’ Elastic Beanstalk:

.dockerignore
.ebextensions/           # Describes how EB builds environments
  01_envvars.config      # Describes env vars for AWS Docker containers
.ebignore                # Describes how Amazon EB ignores some files
.elasticbeanstalk/       # Location Amazon EB stores its cli settings
Dockerrun.aws.json       # Describes how to run our containers in AWS
.gitignore
.bowerrc
.csslintrc.json
.jshintrc
bower.json
gulpfile.js
package.json
app/
  apps/*
  project/*
  dist/*
bin/*
docker/
  django/
    dev/
      docker-compose.yml
      Dockerfile
    prod/
      docker-compose.yml
      Dockerfile
      gunicorn.conf.py
    start.sh
  nginx/*
environments/
  dev/
    .env
    .env.example
    Procfile
    requirements.txt
  prod/
    .env
    .env.example
    Procfile
    requirements.txt

.ebextensions/01_envvars.config

This file is used by the Elastic Beanstalk command line utilities to pass key-value parameters to Amazon EC2 and ECS. This file is used to store all production environment variables that are provided to running containers. This variables often vary between deployments.

.ebignore

When Elastic Beanstalk does a deployment, it creates a zip file of the current directory, uploads it to Amazon S3, and deploys the files to running EC2 instances. Since this project is using Docker images to package the app, we can ignore most files; with the exception of the app/dist directory— which both “Django” and “Nginx” containers will need access too.

The .ebignore file is used to ignore certain files in a project directory. This file works like a .gitignore file.

# Ignore everything!
*
# Except for these exclusion patterns required by Amazon ECS
!Dockerrun.aws.json
!.ebextensions/*.config
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml
!app/dist/**
!docker/**

The only files our EB package should contain are those required by Docker, Elastic Beanstalk itself, or any files shared between both containers (such as the app/dist directory).

When you deploy your project directory to Elastic Beanstalk and create a new application version, the EB CLI will not include files specified by the .ebignore in the source bundle that it creates. This is useful for creating smaller packages by excluding files that aren’t required for running production-only code.

.elasticbeanstalk/

Elastic Beanstalk uses this directory to store temp files and configuration information about the current AWS account, EB Application name and IAM credentials to utilize.

Dockerrun.aws.json

Dockerrun.aws.json is a proprietary Amazon-specific JSON format called a “Task Definition” used to configure how to manage Docker containers running on Amazon EC2 Container Service (ECS) platform.

"containerDefinitions": [
{
    "name": "django",
    "image": "0xadada/dockdj:latest",
    "essential": true,
    "memory": 512,
    "mountPoints": [
        {
            "sourceVolume": "gunicorn-conf",
            "containerPath": "/etc/gunicorn/gunicorn.conf.py",
            "readOnly": true
        }
    ]
}

The JSON format is very similar to the docker-compose Yaml format, having a nearly 1-to-1 mapping of image, mountPoints to volumes and ports all defined.

This file is functionally identical to docker/prod/docker-compose.yml in that it runs, configures and connects the “Django” and “Nginx” Docker containers. As such, changes to the docker-compose.yml file should be mirrored in the Dockerrun.aws.json file.

Lifecycle

New developers to this project simply clone the project from GitHub, install Docker (and boot2docker/docker-machine on OS X) and can begin running the app. There is no need to setup a developer environment or create (yet) another VM.

Development

When the developer starts working on the project from scratch, the only requirement is Docker and a machine capable of running Docker containers (Linux 3+ or boot2docker/docker-machine).

Once an organization or developer has adopted Docker for a single project, startup time for other docker projects is drastically reduced as this core requirement has already been met. From that point forward, the projects themselves can define and provision their own dependencies.

For this project, the next steps required of the developer are as follows:

git clone <PROJECT>
<create .env file>
.bin/stevedore dev start

The developer is now running the app. Any internal OS configuration, system libraries, software dependencies and provisioning are all handled by the project and Docker— transparently to the developer.

Subsequent context-switches between other projects and this project have been reduced to a single command:

.bin/stevedore dev start

The developer doesn’t need to boot up a VM, nor does she/he need to understand or start any internal processes or run any commands internal to the VM.

In development, Docker can be used to lower cognitive load on developers switching between multiple projects.

QA

In this particular project, QA tests are run during build of the the production Docker image via the gulp build task. See the production Dockerfile to view how it calls the gulp task.

In effect, this will prevent developers or continuous Integration systems from publishing a production Docker image to Docker Hub, as the build will trigger a Docker image build failure.

More generally, development teams could create different tags for “production” releases and “development” Docker image releases. Lets say “prod” vs “dev”.

Development teams could publish images tagged with “dev”, to Docker Hub. Other developers on the team or members of the QA team could docker pull that tagged image and run their suite of tests on it.

Using Docker in this manner, dev and QA teams no longer have to keep VM configurations synchronized, as the OS and other dependencies have been pushed down from VM directly into dependencies within the scope of the project. This has the effect of reducing manual synchronization and de-necessitating out-of-channel communication between development and QA teams about the state of the runtime environment. This allows for faster, less-error-prone iteration of the runtime environment.

Production

Developers have iterated on functionality, QA has run tests against the code, and the projects is ready for deployment to production.

At this point, a working Docker image has been run on developers local machines, and QA has passed. These three phases could’ve gone through multiple iterations while bugs were identified and fixed. The end result is a working Docker image exists that has been deemed “ready” for production.

Either manually or as part of a continuous integration tool, the production-ready Docker image can now be tagged with a release version and published to Docker Hub (or other compatible Docker image repository). Finally, the deploy process needs to update the production servers running our working application stack and run the latest application code.

These tasks are handled by our bin/deploy script, a wrapper for Docker, Git and Elastic Beanstalk. It will tag the latest Docker image, publish the tag to Docker Hub, tag the publish the tags to GitHub and use Elastic Beanstalk to deploy both the latest stack and application code:

(Lets use 1.2.3 as an arbitrary version number for this example)

bin/deploy release 1.2.3 # Create a release branch and tag the image
bin/deploy publish 1.2.3 # Publish the Docker image and git branch
                         # to Docker Hub and GitHub
bin/deploy deploy 1.2.3  # Use EB to deploy the latest release

The deploy script is a light bash wrapper that automates Git, Docker and Elastic Beanstalk commands in an easy-to-reproduce set of short commands.

Once complete, the Amazon environment will be running your latest application code, as well as any new changes to the container OS, system libraries and dependencies. Most importantly, any changes in provisioning to the stack have been deployed along with the Docker image, thus enabling seamless roll-backs. Rolling back the application version will also rollback the stack version. The application and stack are deployed together.

Happy cloud computing!

Terms

• Auto-scaling A method of setting a threshold that detects when the load on a server cluster necessitates adding or removing servers in order to optimize the number of servers servicing that load. Auto-scaling allows an organization to decrease operating costs by running the minimum number of servers required to service its load, and eliminating the need to accurately predict future traffic patterns.

• Configuration Management Software tools that are designed to automatically start, provision and configure software on virtual machines rather than have engineers run these steps manually on each server. These tools can be used both locally to create development VMs (virtual machines) as well as in the cloud to create staging and production VMs.

• Docker image A docker image is like an executable program binary. It takes source files and other assets and bundles them together, and the resulting bundle can be run/executed as a single process on a Linux machine.

• Docker container A docker container is like a running executable program. It is a running instance of a docker image. Like a running program, it has a PID, and it is appropriate to call it a process. It can be started and stopped. One docker image can be run many times on one or more machines.

• Provisioning The installation and configuration of software needed to run an application. E.g. Installing and configuring Apache and its system libraries in order to run WordPress.

• Task definition A proprietary JSON format for describing how Docker containers are run within the Amazon EC2 Cloud Service. Read more about Amazon ECS Task Definitions. Docker uses the docker-compose Yaml file to do the same thing.

The RASpec Impreza is a 2002 Subaru WRX “bugeye” chassis purchased in 2011 and fully restored from 2013 to 2015. The car was built by DENT Sport Garage and German Performance Service in Massachusetts.

I built the RASpec Impreza as a cross-campaign race car capable of competing in time attack, time trial, hillclimb and wheel-to-wheel racing series up and down the East Coast.

This car was originally purchased in 2004, and gradually transformed from street car into full race car. The build was done iteratively informed from performance metrics at races.

Chassis

The chassis is a 2002 model year, GDA/B code.

The restoration began with a complete tear-down. Everything not welded to the chassis was removed, tagged and inspected. Anything not required for racing such as the interior and OEM suspension was sold. Anything worn was cataloged, and new factory OEM replacements were ordered (such as chassis gommets and rubber window seals). The paint and rubberized under-body weatherproofing was removed by media-blasting and sanding. Next the chassis was fully seam-welded. Finally the roll cage was built and the car was freshly painted white.

• WEIGHT: 2762 lbs
• CAGE: SCCA multi-point rollcage
• Seam-welded chassis

Speedhunters

Engine

The RASpec Impreza engine is a JDM EJ207 WRX STI 2-liter engine running the Tomei 2.2l stroker kit. The Tomei 2.2l stroker kit was chosen as it is complete with pistons, rods and the lightened, custom journaled crankshaft. The heads are sourced from a JDM Spec-C WRX STI that come from the factory with larger ports, and factory polished.

The engine was built to tolerances provided by the team running the Cusco / Tomei / Voltex Time Attack Impreza as they use the same engine stroker kit.

The intake side has a custom fabricated front mounted intercooler (design is inspired by the Subaru Impreza WRC S10 design) pressurized by a Garrett GT3076R turbo using a TiAL GT30 turbine housing to feed the reverse mounted JDM STI intake manifold (no TGVs).

The turbo wastegate is a TiAL MR-R 44mm. The turbo uses the TiAL V-Band Clamp and Flange System to allow full 306 degree rotation for optimal turbo positioning.

• JDM WRX STI EJ207 block
• JDM WRX STI Spec-C heads (Ported & polished)
• JDM s202 Spec-C valve covers (wrinkle red)
• Tomei forged pistons 92.5x79mm (2.2l stroker kit)
• Tomei forged rots (2.2l stroker kit)
• Tomei balanced crankshaft (2.2l stroker kit)
• Tomei bearings
• ARP head studs
• Cosworth 275 degree cam shafts
• Spark plugs NGK BKR7E (stock #4644)
• Garrett GT3076R (A/R 0.82)
• Tial MV-R 44mm external wastegate
• TiAL GT30 Turbine Housing
• Cosworth oil pump
• Oil starvation baffler
• Mocal oil cooler
• Crawford Performance Air/Oil Separator
• Koyo aluminum radiator
• Griffin front mount intercooler
• Stainless custom exhaust by DENT Sport Garage

Transmission

• 2006 Subaru WRX STI 6MT
• ACT 6-puck clutch
• ACT ProLite flywheel
• STI axles
• Cusco transmission mount brackets
• Cusco transmission hard mount bushings
• Center diff controller DCCDPro Delux DCCD controller
• Front LSD: STI
• Center LSD: STI
• Rear LSD: Cusco RS 1.5-way
• Kartboy short shifter linkage kit

Suspension

• Stance GR+ 3-day adjustable coilovers
• Cusco 3-way adjustable rear sway bar
• Pillowball swaybar end links
• Whiteline adjustable rear lateral links
• STI Front control arms
• 6Gun Racing Ball joint extenders w/ tie rod ends
• ARP Long wheel studs
• Whiteline control arm anti-lift kit
• VOLK CE28N Time Attacks 17” x 9” 5x114.3 +30
• PF Performance Friction 2-piece floating rotors w/ aluminum hats
• PFC Brake pads (#0460.01.15.44 132mm x 61mm x 16mm)
• Stainless steel braided brake lines

Fuel

• SARD Fuel surge tank (trunk mounted)
• RC Engineering 1000cc injectors

Electronics

• Vi-PEC ECU
• Ballenger Motorsports AFR500 Wideband O2 Air Fuel Ratio monitor kit

Exterior

The exterior is stock, with a few OEM components. The front bumper is running the “Double Lip Combo” setup. The upper is the 2002 Prodrive V2 lip and the lower is the 2006-2007 V-Limited lip.

• COLOR: “Audi R8 White”
• Scoote carbon fiber 2002 WRX hood
• Aerocatch hood pins
• Prodrive “V2” 2002 WRX lower front bumper lip
• 2006 Subaru STI V-Limited lower front bumper lip
• Rear wheel fenders are rolled & pulled to accommodate larger
• STI carbon front blinker covers

Interior

The interior is fairly spartan and without adornment. Only the essentials.

• MOMO steering wheel
• BRIDE Zeta-3 (Red)
• Defi BF-series Oil pressure, Boost pressure & Water temperature
• Digital A/F Meter
• Emergency battery/ignition cut-off switch
• G-Force 6 point Cam-lock Harness
• NASCAR Window Net (Drivers side)
• Beat Rush trunk bulkhead panel

Spare Package

The car comes with additional spares, hardware and accessories.

• PF Performance Friction rotors (New)
• OEM STI Rear Lateral Links (New)
• 15 gal VP 109 octane race gas
• Origin GT Wing (plane only)

History

I raced the RASpec 1.0 at COM Time Trial events, the Redline Time Attack series, the BMW Sports Car Club track events, and New England Hillclimb series.

A crash at a New England Hillclimb Race Series in 2010 ended the useable life of the “RASpec Impreza 1.0”, and I began searching for a “bugeye” Impreza to use as a new chassis for the “2.0” version of the RASpec Impreza.

In 2011 I found a “bugeye” WRX that would serve as the perfect base for a new race car. The bugeye was parted-out and stripped, and the build of my second race car began. It was only fitting that it be called the “RASpec Impreza 2.0”.

The key features of the 2.0 version that were improvements over the 1.0 version were as follows:

• Fully stripped, media-blasted, caged and seam-welded chassis in order to reduce weight and increase rigidity
• Simplify turbo and intercooler setup to reduce maintenance and increase air flow

The resulting product of the build is a car that is lighter, faster and easier to maintain than the first race car. The “RASpec Impreza 2.0” is a better overall car, it takes the best aspects of the 1.0 and simplifies until there is nothing else to take away— creating a minimalist race-ready track platform.

Significant Results

2006

• COMSCC July - NHMS
  • 2nd Place - ST1 Street Touring 1 (of 5)

2007

• COMSCC Season Standings
  • 3rd place - ST2 Street Touring 2 (of 10)
• COMSCC June Mosport international Raceway
  • 2nd Place - ST2 Street Touring 2 (of 3)
• COMSCC July - NHMS
  • 1st Place - ST2 Street Touring 2 (of 5)

2009

• COMSCC July NHMS
  • 2nd Place - SPB Street Prepared B (of 5)

2010

• Duryea Hillclimb - August
  • 1st Place / 4th overall - Improved Touring E (of 103)

Video

FAQ - Frequently Asked Questions

• What makes the car track-only? Its a fully-built race car, so its been stripped to the bare essentials. This means no headlights, no wiper bottle, no seatbelts (harness only), no OEM seat brackets, no ODB-II.
• What does it need to go racing? I would get new tires, bleed the brakes and adjust the alignment. Alignment should be set to your preferences for how you like the car to turn-in and rotate. You may want to adjust the ride-height as well.
• How many track miles on it? None. The chassis has 0 track miles on it, and the engine has been freshly rebuilt. The only “time” its seen is on the dyno while it was being tuned.
• Horsepower? Currently it’s running a conservative base map dialed for 350 wheel horse power at 20lbs of boost. This tune was done at Synaptic3 Performance for a practice day where we were focusing on suspension setup and were not racing. The previous tune made 400whp at 23lbs of boost. On dyno testing its made much more at higher boost, but we’ve dialed it back to increase engine longevity. Using the same engine parts, tolerances and turbo, Cusco / Tomei / Voltex made 500whp.

2019 Update

This car is now owned and campaigned at Global Time Attack and other events by Bad Panda Motorsports.

The task we need to accomplish is to determine which workflow is right for your project. This article will attempt to provide an overview to the different workflows, and explain when merge and rebase are the more appropriate workflow to use.

The Simplistic Branch Workflow

aka the GitFlow / Branch-Merge-Merge Workflow

Software development teams have a typical way of doing things, but all have a main integration branch (I’ll call it “Master” for the remainder of this article) that receives feature branches.

Feature branches originate from Master, active development is performed along the feature branch, and when the feature is complete, it’s combined back into Master.

I call this workflow the “GitFlow / Branch-Merge-Merge” workflow. The history of events for this simplistic workflow are as follows:

on Master:

• commit #1
• commit #2

on Feature:

• commit #a
• commit #b

on Master:

• merge feature --no-ff

This produces the following graph:

This workflow performs well for a single developer because while the feature development occurred on the feature branch, the sequence of events occurred linearly. Importantly, no changes occurred on Master while Feature branch was changed.

When development occurs in parallel on the upstream branch (ie. Master), either by the developer, or a co-developer, the history gets more complicated.

For this example, we’ll pretend to be a single developer working on both branches in parallel, we use the following sequence:

on Master:

• commit #1
• commit #2

on Feature:

• commit #a
• commit #b

on Master:

• commit #3
• commit #4

on Master:

• git merge feature --no-ff

This produces the following graph:

In the above scenario the developer runs into the problem where Feature commits #a and #b were done independently of Master commits #3, #4 and #5. This merge will fail.

To solve this problem, the developer must first merge the upstream changes in Master into feature branch Feature. To accomplish this, the developer must switch into the feature branch and merge in the changes from Master. The developer must then switch back to Master and merge the changes from Feature into Master. Use the following sequence:

on Feature:

• merge master

on Master:

• merge feature

This produces the following graph:

I call this workflow the “Branch-Merge-Merge” workflow because in the most general case — the workflow begins with a branch, requires an upstream merge into the feature branch prior to merging the feature branch back into the master branch.

Looking at the above graph, feature branch Feature has been merged successfully into Master, but the git log shows an interleaved sequence of commits between Master and Feature that make debugging complex.

In the case where branch Feature introduced a bug into the Master branch, a rollback will be difficult because of how the commits are interleaved. In the case where the bug was introduced in Feature branch commit #b, the maintainer of Master branch would need to roll-back commits #6, #c, #5, #4 and #b. This is a difficult roll-back because Master branch features #5 and #4 were rolled back in the effort to roll back to Feature branch feature #b. We’ve thrown some of the baby out with the bathwater.

One way to address the problems with this particular scenario is to merge in the Feature branch as a single commit appended to the Master branch, rather than as a set of interleaved commits within the Master branch. This brings us to the next workflow.

Branch-Merge-MergeSquash Workflow

This workflow attempts to address the shortcomings of the workflow above by appending feature branches as a single commit at the end of the integration branch.

For this example, lets pretend to be a single developer working on both branches in parallel (as I did above), we use the following sequence:

on Master:

• commit #1
• commit #2

on Feature:

• commit #a
• commit #b

on Master:

• commit #3
• commit #4

on Feature:

• merge master

on Master:

• git merge feature --squash

This produces the following graph:

Looking at the above graph, feature branch Feature has been merged successfully into Master (although the graph doesn’t show a merge line) at commit #5. Importantly, the git log shows a linear sequence of commits, each indicating a complete feature integration. The squash flag has collapsed Feature branch commit #a, #b and #c into single commit #5.

In the case where branch Feature introduced a bug into the Master branch, a rollback will be as simple as running git reset --hard HEAD^ because of how the Feature branch was a single commit appended onto Master. In this scenario, the Master branch maintainer would tell the Feature branch maintainer to fix the problem, and they could attempt the merge once the issue was fixed.

I recommend this workflow as a best-practice in cases where the feature branch will be shared (eg. pushed to GitHub, shared with other developers.)

A potential downside of this workflow is the potential confusion caused by the origin commit of the Feature branch. Feature branch origin is located at commit #2, and merged at commit #5. This is additional cognitive load on the Master branch maintainer when it comes time to merge the feature branch into Master. A simplified workflow could solve this by having the Feature branch simulate its origin commit at commit #4 prior to merging into Master. This would reduce cognitive load on the Master branch maintainer.

Branch-Rebase-MergeSquash Workflow

This workflow is popular with open source projects where the job for the Master branch maintainer involves multiple feature branch integrations on a daily basis.

This workflow is designed to make feature branch merges much simpler and easier to roll back. The main idea is that each feature branch is a simple append to the end of the Master branch. No merge is simpler than and append-merge.

For this example, lets pretend to be the same single developer working on both branches in parallel (again, as I did above), we use the following sequence:

on Master:

• commit #1
• commit #2

on Feature:

• commit #a
• commit #b

on Master:

• commit #3
• commit #4

on Feature:

• rebase master

We have now changed the origin commit of branch Feature to commit #4. This sets up future merges of the Feature branch to append directly to the end of Master branch.

on Master:

• merge feature --squash

We’ve now switched to the Master branch, and merged in the rebased Feature branch, appending a single commit #5 to the end of the Master branch containing all of the Feature branch.

This produces the final graph:

The final sequence of events on the Master branch read cleanly, showcasing only past feature branch integrations. Each feature branch originated directly from the latest commit, making it seem like the feature is a simple fast-forward append onto the latest commit.

It’s worth pointing out the Branch-Merge-MergeSquash and Branch-Rebase-MergeSquash workflows both result in the same final sequence of commits. The major benefit of the Branch-Rebase-MergeSquash workflow is that its’ branch origin appears much later in the history, reducing cognitive load on Master branch integrations.

I recommend this workflow as a best-practice when the developer is working alone on a feature branch, as well as just-prior to merging the feature branch into the master branch.

The downside of this workflow is when sharing a feature branch after a rebase. Pushing to GitHub will require the --force flag, and anyone else working on the feature branch will see changes to the branch history when they pull an update. This is why I don’t recommend this workflow for shared feature branches.

Conclusion

I see huge benefits in collapsing feature branch commits into the master branch with — — squash, both for simplicity sake, clarity, and reduces cognitive load when the master branch maintainer needs to merge in big features. Additionally it makes the use of git bisect much easier.

I love the advantages to using git rebase to merge in upstream changes while working on a feature branch, but it seems people need to use the workflow in practice before they can understand its benefits. Give it a shot, you might like it.

All three workflows have been published to GitHub

• Branch-Merge-Merge
• Branch-Merge-MergeSquash
• Branch-Rebase-MergeSquash

This article was originally published on Medium.com

This seemingly benign question is steeped in the cultural heritage of the New England colonial Puritan/Protestant work-ethic. Hard work, frugality, perseverance in the face of hardship (without complaint) is the mark of Grace and this will allow you to achieve salvation. You are your work, and your value and place in the social hierarchy is defined by your job. There is a built-in assumption that It’s also a uniquely American question to ask. In Europe you don’t hear this question asked at social events—work doesn’t define their lives outside of work, and frankly, it is impolite to discuss work when there are more interesting things to talk about.ones job defines ones personal identity.

This ideology sustains the basis for capitalistic inequality. The work-ethic creates two groups of people: those who work hard and are deserving salvation and those who do not. In defining the qualities of a ‘Good’ person, its’ negation reveals those who are unmentionable; the ‘Bad’ person: the sinner, the lazy, the thief. There is a third group outside of the unmentionable, the unmentioned: those too young, too old or unable to take part in work, those prevented from work, those who don’t work, and those who don’t believe in the work-ethic.

In asking the question, two assumptions are made:

1. The other person has a job
2. For many people, this simply doesn’t apply.Their job describes their identity to some extent.

During conversation, this question can be interpreted as a strategic move by the interrogator. In asking first, the interrogator is positioned to answer the counter-question with a more strategic response. This response can establish the upper hand in the contextual social hierarchy. Much like claiming higher ground prior to battle. It can come off as setting a stage to brag about ones job. Those conversations tend to unfold something like this:

Roy: “So what do you do?”

Steve: “I work at a red-sauce joint in Brookline” (see note)

Roy: [Puffs his chest] “I’m the Director of Technology at EvilCorp”

When asked “what do you do?” I try to From the French détournement, a rerouting, to misappropriate, to hijack on purpose. Turning expressions of the capitalist system against itself.detour the question with some unexpected and obsolete profession I run a VHS rewinder service. For one, most people don’t care about the answer, they ask out of cultural habit. If pressed about my job, I usually say “I make internet stuff” and explicitly leave any job title or hierarchy out of the response.

A better question to ask people in a social setting:

Who do you know that I know?

This question has a goal, namely to create a real connection between people that exists beyond bullshit corporate jargon and hierarchy.

If we want to move beyond a society structured by hierarchy and class, we need to think outside our own ideology before we speak.

Worse yet, if users are able to find the browser preference, they don’t understand its purpose or value. Enabling the feature is confusing, and quite inconsistent between browser vendors.

Goggle Chrome hides the feature behind an “Advanced” barrier. Features and settings are typically placed behind this type of barrier when the feature could cause problems for novice users, or as a catch-all for rarely used features. Features labelled “Advanced” intimidate many users from activation.

Chrome labels the feature: Send a “Do Not Track” request with your browsing traffic which is misleading; an additional request is not sent, instead a header is included in requests made to the server.

Chrome further discourages activation with a confirmation dialog containing a wall of legalese when the user clicks the check box. see video illustration

Mozilla Firefox features the DNT setting more prominently at the top of the Privacy tab. While the wording is simple and clear, it offers little contextual explanation, and provides no incentives to encourage activation.

Apple Safari is the most confusing, it labels the DNT feature with the text “Website tracking” followed by a check box. This construction misleads users into thinking that enabling the feature will act as an explicit opt-in to website tracking. Ask websites not to track me is a request to opt-in to a negative; this requires a moment of mental aerobics many users trip over.

This of course, discourages users from enabling the setting.

Microsoft made a bold choice by choosing to enable DNT on by default in Internet Explorer 10. A great example to set, and good for their user privacy. The wording of the feature label could use some clarification however.

Again, the wording here misleads users into thinking that additional requests are made to sites they visit.

DNT is not a technical problem

At its core, this is a social problem, not a technical problem. There are four stakeholders involved, all their interests need to be aligned in order for this project to succeed. The user needs to understand the value of DNT to their everyday privacy. The browser vendor needs to be encouraged to implement DNT that best serves their interests: acquiring & maintaining users, and accommodating content providers.

If DNT is going to succeed, the approach needs to give incentives to both browser users as well as content providers.

The Approach

The most effective way to promote the adoption of DNT is to appeal to peoples emotions.

Lets first consider the appeal to fear. An appeal to fear will result in web browsers who are afraid of marketing companies building “Shadow Profiles” behind their backs and black-masked hackers around every corner looking to steal their identities. An appeal to fear will result in content-providers fearing the day search engines begin punishing them for failing to honor DNT.

The alternate approach is to appeal to positive emotions. Highlight the user benefits of DNT as a browser feature that promotes privacy, while highlighting the business benefits of DNT to content providers, search engines and browser vendors.

• Reward users by showcasing what they gain by enabling DNT.
• Reward browser vendors who implement DNT in a user-friendly way.
• Reward content providers that honor DNT with increased search engine position.
• Reward search providers by increasing or maintaining user return rate

Users

Users will be empowered when they’re given tools to control their online privacy. This is initially an emotional feeling when their browser displays a message in the UI indicating their privacy is more protected, but more long term it maintains free agency with ones interaction with the web.

Browser Vendors

Browser vendors will be motivated to increase user acquisition and maintenance as users switch to the browser with more simple and easy to-use privacy-centric functionality.

Browser vendors should increase DNT usage by defaulting to DNT=1 when the user enables privacy mode.

Content Providers

Sites offering pro-DNT content and services will acquire more visits with higher search engine placement; once Search providers begin rewarding sites that honor DNT.

Search Providers

As users migrate to search providers that offer pro-privacy services, these providers will see an uptick in user acquisition.

There is an emerging privacy market, and those with pro-privacy offerings will eventually be the ones who win — as marketing surveillance becomes more ubiquitous and users wise-up to its encroachments on privacy.

Updates

January 2019: Apple Safari browser has removed support for the expired Do Not Track standard to prevent potential use as a fingerprinting variable. Tracking Protection Working Group has been closed.

Resources

• Mozilla DNT Usage Metrics Dashboard
• W3C Unofficial Draft Proposal
• W3C Tracking Preference Expression (DNT)
• FTC Comment on DNT
• Do Not Track Field Guide (Mozilla)
• Do Not Track Implementation Cookbook

Thanks

Thanks to @MickD for informing me that Microsoft IE10 comes with DNT enabled by default.

This article was originally published on Medium.com

Another characteristic reversal after passing a road break boundary is that the country ceases to be the center of all work, and the city ceases to be the center of leisure. In fact, improved roads and transport have reversed the ancient pattern and made cities the centers of work and the country the place of leisure and of recreation.¹

He goes on to describe how farmers lived and worked on the land in the countryside. The rise of industrialization lead to a shift in the focus of labor activities to the factory and centers of production around the factory — the city or urban area. The invention of the train helped to leverage the growth of the cities by facilitating transportation between centers of production for both resources and labor. Similarly, the rise of the automobile and interstate highway systems directly resulted in the development of suburban areas.

McLuhan utilizes the agrarian-to-industrial economic shift as a template by which we will see the next shift; from an industrial economy to an information economy.

The (in his words) “electronic economy” or information economy (in contemporary parlance) will be a new economic/societal shift in which the information infrastructure (fiber optics, cheap hardware, ubiquitous internet access) will afford another shift from the centers of industrial production back to the countryside — ie: working remotely from your country home.

McLuhan, in the dawn of the color-TV era, was widely seen as a visionary capable of seeing deep into societal shifts before they happened, and providing guidance as they unfolded. Perhaps his works should be revisited, as they continue to provide useful insights.

The USD is no longer backed by gold¹, but by a promise that the US Government will pay its debt (in the form of payment the interest on (and sometime redeem) government securities when due). If these payments are not made to the debt holders (Corporations, China, treasury-bond holders) then a default would occur, which would shift international use of the USD into other currencies. This would flood the US economy will low-value dollars, resulting in massive hyper-inflation.

If you remember the the “Debt Ceiling Crisis” of 2011, the “Fiscal Cliff” of 2013 (and the Government Shutdowns of 2018 and 2019)², then i’m sure you can understand why international faith in the US Gov promise to pay its debt is wearing thin.

The USD is also backed by simple faith in that the dollar is “Just what everyone uses, so I’ll use it”. This is fine and good. Many currencies have succeeded since the early 18th century largely on faith. Faith can shift based on perceptions however, so once people begin to see that “The Emperor Has No Clothes”, they will begin to store value in other stores-of-value.

So what is Bitcoin backed by?

So basically BTC is backed by mathematics, technology and an economy, just like any other currency, but without a central bank. Some argue this is better than any other fiat currency because a fiat currency is controlled by (and manipulated by) a governing power; BTC is peer-to-peer. The block chain is the public record of all BTC transactions, in chronological order. This means that all transactions are known to the world; contrast this with how little we know of central bank activities.

Mathematics Bitcoins are created and transacted by a mathematical proof-of-work³. This means that in order to create a new bitcoin, and to conduct a bitcoin transaction; computers must provide a solution to a complex mathematics problem that takes time & electricity to compute. This creates intrinsic value in the form of time and cost-of-electricity and simultaneously verifies the authenticity of the new coins and transactions. Contrast that with the intrinsic value of a piece of printed paper of a US Dollar bill.

Economy Bitcoin is “backed” by its usefulness. Just as is the US dollar. As long as you can do things with it, then it has value. The more people can do with BTC, the more value it will have. In time, the technology and economic rules might make BTC more appealing than the USD.

• Write your CSS selectors to be as concise as possible
• Write your CSS selectors to be as performant as possible, efficient as possible and even more efficient
• Don’t overqualify your CSS selectors
• Use the lessons learned in “Object-Oriented” CSS

Make use of the child selector to limit how deep your css goes. .nav-list > li can do a lot to prevent the cascade from going too deep.

Think of the class attribute in HTML as a mounting-point for CSS classes, fixtures upon which styling is attached to markup. This is opposed to using the markup as a skeleton on which you drape your style.

When asked to change existing code, rather than simply adding complexity with new classes or additional selectors, can you refactor or abstract existing selectors or classes to produce the desired behavior? Look at change requests as an opportunity to refactor.

Code reuse is your friend. The first “C” in CSS stands for Cascade. Use the cascade to maximum advantage. Inheritance is extremely powerful. When adding a css property, ask yourself if you can add it higher in the cascade, and inherit it from a higher abstraction.

Write obvious code. CSS is fragile, write code as if a novice coder would be maintaining your code in the future.

”Extent your CSS objects with class” - not with selector specificity. Increasing selector specificity makes overrides more difficult and fragile to maintain. Class names can always be more semantic than the generic tag names at your disposal.

Browsers actually have a very similar set of core default-styles. Rather than obliterating the core-styles with a hard-reset css, think about the value of css normalization. Using a normalized approach, the focus is to normalize the differences across browsers rather than eliminating all styles together. This means you spend less time re-creating the core styles that are very useful in practice.

Create a UI Pattern and Component Library - MailChimp has a good one.

Elements that are effected by javascript behaviors should have css modified by adding or removing classes. Keep your css in the css-file, don’t pollute the javascript with css properties when you could abstract those properties to the css file. Elements that have multiple states should have corresponding class names that describe those states when added to the element. The default state of the element is when it has no additional descriptors in the class attribute - the default state. For an anchor tag button, some additional states class names could be:

is-active
is-disabled
is-hover

General High-Order Principles

• Simplicity - The most bug-free line of code is the one you don’t have to write!
• Reusable - Create code that is portable to other sections of the site and are nestable
• Extensible - Create code that lends itself to be easily extended and with low selector specificity to promote extensibility.
• One-to-many - Create classes that can be applied to many contexts
• Fast - Know your selector performance basics, reduce HTTP requests
• Semantic - Create CSS that speaks to the task or meaning that is being accomplished. Good CSS class names describe our HTML elements in very semantic and specific ways
• Standards Oriented - Create portable, standards-forward code (ie Don’t use hacks)
• Future Proof - Write code as if you’ll never be able to edit it again
• Accessible - Just as people have different capabilities with respect to vision, hearing, language or cultural understanding, devices have varying levels of memory, CPU, screen size or bandwidth
• Progressive - Create a core experience for the least capable devices (legacy devices, low bandwidth, low power, small memory, small screen) and progressively enhance the experience as you detect richer capabilities
• Fast to develop - Write code that is fast to create new features, extend existing features, and maintain old features
• Team scalable - The code and conventions should be readily apparent to a novice developer who joins the team, and easy to adopt so he/she can quickly contribute

Strive To

• Separate Structure from Skin
• Separate Skin from Behavior
• Separate Container from Content
• Create a Component Library - Use Components like legos, combining them to create rich UI widgets. Create a reference page to see all components in a single centralized place
• Separate components into generalized sub-groups: (Ordered from least to most specificity)
  • All - The manifest that imports all other stylesheets
  • Settings - Global variables, public mixins, config switches, brand colors, etc
  • Base - Base styles that normalizes globals, everything inherits from these defaults
  • Objects - “o-” OOCSS design patterns, reusable UI primitives
  • Components - “c-” Fully designed pieces of UI, Still only utilizes class selectors. More expliciltly named (e.g. products-list)
  • Trumps: Overrides, only affect on piece of the DOM at a time. Usually carry !imporant.
• Your markup should be distilled from combining your component library pieces together
• Minimize the amount of bespoke or one-off css-possible
• Extend Components by applying abstracted classes
• ”Flow like water” - Be flexible, allow site to respond to content to dictate height, and screen size to dictate width. Layout should respond proportionally
• Use themes to separate ornamentation from core design
• Increase portability with context-free CSS

Separate Structure from Skin

Focus on keeping your markup semantic. Don’t write markup to achieve a presentation-level effect, leave that to the CSS.

Separate Skin from Behavior

Use CSS to skin your markup, but further separate your CSS from behavior. If we use an anchor tag that acts as a form-POST event, write two classes; one to describe the look of the button and another as its hint to javascript that’ll be used for attaching functionality.

Markup

<a href="/action" title="Submit" class="c-button js-action-submit">Submit</a>

CSS

.c-button {
  color: green;
  border: 1px solid red;
}

JavaScript

$( '.js-action-submit' ).click( function( e ) {
  e.preventDefault();
  // do form submit
}

Use themes to separate ornamentation from core design

Gradually tastes change like the seasons. The design team will continue to evolve the graphic design as well as to create new design ornaments. Rather than continually editing a large group of styles to integrate these changes, strive to limit ornamentation to a set of core “theme” classes that are applied to elements. Avoid editing a fragile selector that includes layout and positioning properties that could potentially break the site and instead limit your changes to a specific selector that focuses on the ornamental effects of the element. An example helps:

Markup

<!--Okay-->
<a href="/contact" class="c-fun-button " title="contact us">Contact Us</a>
<!--Better-->
<a
  href="/contact"
  class="c-contact-button t-contact-button---fancy"
  title="contact us"
  >Contact Us</a
>

CSS

/* Okay */
.c-fun-button {
  display: inline-block;
  padding: 1em 0.5em;
  border: 1px dashed red;
}

/* Better */
.c-contact-button {
  display: inline-block;
  padding: 1em 0.5em;
}
.t-contact-button---fancy {
  border: 1px dashed red;
}

Increase portability with context-free CSS

In striving to make components and elements that are easily moved, extended and maintained; reduce the context that your css requires. For this to work, your component must have:

• A unique name (css class) that will be part of the css selector
• No HTML element decendant selectors ( .menu li ) as these are inherently context-dependent

Instead, use the unique name for the parent elements css selector, and separate the child elements css class names with two – hyphens, for example:

Markup

<ul class="c-navmenu">
  <li>
    <a class="c-nav-menu–-item">an item</a>
  </li>
  <li>
    <a class="c-nav-menu–-item">an item</a>
  </li>
</ul>

CSS

.c-nav-menu {
  property-name: property-value;
  property-name: property-value;
}
.c-nav-menu–-item {
  display: inline-block;
  font-weight: bold;
  color: red;
}

In this example, the menu items can be moved independently to other parts of the site. And while this example might not occur in the real-world, it would work- functionally-speaking.

If you need to extend the item to introduce a slightly different style, you duplicate the class name, but add a modifier class separated by three — hyphens.

CSS

.c-navmenu-item---light {
  font-weight: normal;
  color: yellow;
}

Markup

<li>
  <a class="c-navmenu–item c-navmenu-item---light">an item</a>
</li>

Extending our already-existing style by adding classes additionally gives us a reusable class that we can then use anywhere we use the original class.

Avoid:

• Tag Specific selectors - Poor portability & reuse, unintended elements inherit unwanted css properties
• #ID selectors - Poor selector performance & reuse, difficult to override due to high selector specificity
• Alpha transparencies on top of irregular backgrounds - Poor rendering performance
• Fixed “height:” Content is dynamic, changing, and heights will change. Try to avoid height, there are many ways to avoid it
• Fixed “width:” Layouts should indicate proportionality and fit accordingly inside their parent containers. Use box-sizing
• Relying on a defined HTML structure to drape your styles, markup changes over time, and requirements change, components move
• Using javascript to modify css properties - don’t pollute your javascript with css

Adding Behavior

Class names are often used to add a behavioral hint that our javascript will use to select an element and add some sort of behavior, ie: a tab control made up of an unordered list with anchor tags inside list elements. Clicking the anchor tag results in another container element showing or hiding. Separate the behavioral hint by using a class name that indicates its action.

js-action-apply
js-action-navigate
js-action-cancel
js-action-next
js-action-play
js-action-toggle
js-action-clickable
js-action-draggable

Avoid using these class names to apply css styles.

Normalize vs Reset CSS

We have moved away from Eric Meyer’s reset.css to normalize.css for the following reasons:

1. Normalize.css preserves useful defaults rather than “unstyling” everything. For example, elements like sup or sub “just work” after including normalize.css (and are actually made more robust) whereas they are visually indistinguishable from normal text after including reset.css
2. Normalize.css corrects some common bugs that are out of scope for reset.css. It has a wider scope than reset.css, and also provides bug fixes for common problems
3. Normalize.css doesn’t clutter your dev tools. A common irritation when using reset.css is the large inheritance chain that is displayed in browser CSS debugging tools. This is not such an issue with normalize.css because of the targeted stylings
4. Normalize.css is more modular. The project is broken down into relatively independent sections, making it easy for you to potentially remove sections (like the form normalizations) if you know they will never be needed by your website
5. Normalize.css has better documentation. The normalize.css code is documented inline

For more reasons and rationale, check out the details About normalize.css.

To get an idea what #3 means “Normalize.css doesn’t clutter your dev tools”, as well as pollute the CSS cascade with unnecessary rulessets, checkout the following example. In this image we compare the CSS rule chain for the “About Us” title on the left with reset.css with 18 rules. On the right is normalize.css with 11 rules.

As you can see, the ruleset on the right is far simpler, and unnecessary complexity is always bad.

Further Reading

• bem.info - Block-element-modifier approach to css class naming conventions.
• OO-CSS - “Object Oriented” CSS
• Overqualifying Your Selectors- Writing efficient CSS selectors
• Mozilla - Efficient CSS- Mozilla article on fast CSS performance
• CSS Tricks - Efficient CSS Rendering

”You can measure software maturity by the number of UI toolkits it implements.”

I’ve counted four in Adobe Photoshop CS5.

Travel opens my mind to new perspectives and allows me to recontextualize existing systems of thought.

I had casually studied Roman history over the course of a few years, prior to my travels to Italy. My understanding was built upon a small collection of books and two excellent podcast series. While one can learn a great deal from books, podcasts, and other media, that knowledge is divorced from its original context—i.e. we musn’t mistake a map for the territory.

A museum—by its very nature—strips the subject of its contextual time and place (arguably the time, and sometimes place have been lost to time). A museum succeeds if it is able to recreate the context of the subject in both time and place, within the mind of the museum visitor. This leads to a more delightful understanding and appreciation of the subject.

When I travel, I seek to create new understanding—a recontextualization of the artifacts of knowledge I grasp on the subject. From facts and anecdotes taken from books, images and perspectives gathered from museums—all mashed together again, and then reformed within the milieu of their original spatial context.

Its important to note that this is a passive activity, simply walking around Rome was enough to ignite this recontextualization. Characters take on new roles, events have more (or less) relevance, and places are imbued with life, color and motion. Understanding becomes alive.

Travel seems to make me step outside the world, so that I can look back upon it with new eyes. And when I return home, my new eyes are the eyes with which I’ve always viewed the world.

From comic book stores to Facebook to message boards, it seems that the “Keep Calm and Carry On” poster is living a new life as pop-culture cool while people remix its’ message with both quirky or banal messages.

This attempt tries to speak most directly to its underlying message of consumerism, amidst the current climate of the Occupy Movement.

• Legend
• Motion / Normal mode
• Insert mode
• Changes
• Yank/Put or Copy/Paste
• Visual/Visual Block Mode
• Commands
• Files
• Windows
• Tabs
• coc.nvim
• Denite
• Help
• Search/Replace
• Find Files

Legend

Motion - Normal Mode

• j - Down
• k - Up
• h - Left
• l - Right
• 0, ⇧6 - Line beginning
• $ - Line ending
• e, w - Forward (Small word)
• ⇧e, ⇧w - Forward (big word forward (whitespace only))
• b - Backwards (small word)
• ⇧b - Backwards (big word)
• f<char> - Jump to first occurrence of <char>
• ; - Repeat previous jump
• ⇧f<char> - Jump to previous first occurrence of <char>
• t<char> - Jump to position before first occurrence of <char>
• ⇧t<char> - Jump to position before first previous occurrence of <char> in the direction youre searching
• <n> <motion> - Prefix motion command with a number n to have it jump that n away. e.g. 4li jumps to the 4th ‘i’ forward.
• ⌃f - Forward (page)
• ⌃b - Backward (page)
• ⌃u - Upward (half page)
• ⌃d - Downward (half page)
• ⇧h - Head (of current page)
• ⇧m - Middle (of current page)
• ⇧l - Lowest (of current page)
• gg - Go to top of file
• ⇧gg - Go to bottom of file
• <n>⇧g - Go to line n
• ][ - Jump to next matching brace
• [] - Jump to previous matching brace
• % - Jump between matching braces
• za - Toggle code fold
• zo - Open code fold
• zc - Close code fold
• zr - Open one layer of code folds
• z⇧r - Open all code folks

Insert Mode

These commands are entered while in Normal or visual mode.

• i - Insert at cursor
• ⇧i - Insert at line beginning
• a - Append after cursor
• ea - Append after word at cursor
• ⇧a - Append at end of line
• o - Insert line below cursor
• ⇧o - Insert line above cursor
• x - Delete character under cursor
• ⇧x - Delete character before cursor
• dd - Delete line at cursor position
• d <motion> - Delete amount by motion command e.g. dw delete small word
• c <motion> - Change by motion e.g. cw overwrite small word
• r - Replace character at cursor
• ⇧r - Replace overwriting text
• ^v u <codepoint> - Insert unicode character at codepoint
• ␛ - Exit insert mode

Changes

• ⇧j - Join line with line below
• c(f|t)<char> - Change characters (inclusive with f or exclusive with t) to char

Yank/Put or Copy/Paste

• y - Yank cursor or selection into default buffer
• ”<r>y - Yank cursor or selection into register a (a: can be any letter, e.g. a, b, c)
• yw - Yank word into buffer
• yy - Yank line into buffer
• <n>yy- Yank n lines into buffer
• y <motion> - Yank into buffer by motion
• x - Yank character at cursor
• p - Put default buffer after cursor
• ⇧p - Put buffer before cursor
• ”<r>p - Put value from register a before cursor (a: can be any letter, e.g. a, b, c)

Visual / Visual Block Mode

Visual mode is used for marking text.

• v - Visual mode
• V - Visual line-based mode
• ⌃v - Visual block mode
• gv - Re-select previous Visual block buffer
• o - Jump to other end of block
• ⇧o - Jump to other corner of block
• vaw - Visual select a word (includes whitespace)
• viw - Vissual select inner word (no whitespace)
• vap - Visual select a paragraph
• vip - Visual select inner paragraph
• va[ - Visual select [] block and contents
• vi[ - Visual select [] contents only
• va{ - Visual select {} block and contents
• vi{ - Visual select {} contents only
• va( - Visual select parens block and contents
• vi( - Visual select parens contents only
• va< - Visual select <> block and contents
• vi< - Visual select <> contents only
• va” - Visual select all double quote and contents
• vi” - Visual select inner quoted contents
• vit - Visual select inner XML/HTML tag contents
• vat - Visual select all XML/HTML tag contents
• v<motion> - Visually mark text by motion command
• ␛ - Exit visual mode

The following commands can be used while text is marked.

• c - Delete and begin inserting (To replace currently marked text)
• d - Delete
• y - Yank
• > - Indent right
• < - Indent left

Commands

• . - Repeat previous command
• u - Undo previous command
• ⇧u - Undo all changes to current line
• ⌃r - Redo
• :q - Quit
• :q! - Quit (without saving changes)

Files

• :e <path> - OpEn file into buffer
• :w - Write buffer to file
• :ls - List open buffers
• :(b|buf|buffer) <n> - Switch to buff n
• :(bd|bdelete) <n> - Delete buffer n (close file)
• :bp - Switch to previous buffer
• :bn - Switch to next buffer

Windows

• ⌃wo - make current window the Only window
• :(sp|spl|split) <path> - SPlit window horizontally, optionally loading file at path
• :vsp <path> - Vertically SPlit window, optionally loading file at path
• ⌃ws - Window split (horizontally)
• ⌃wv - Window split (vertically)
• ⌃wx - Windows eXchange, swap their position
• ⌃wk - Move window up
• ⌃wj - Move window down
• ⌃wh - Move window left
• ⌃wl - Move window right
• ⌃wp - Move window previous
• ⌃wc - Window close
• ⌃w= - Balance windows
• ⌃w+ - Grow horizontal window split by 1
• ⌃w- - Shrink horizontal window split by 1
• ⌃w> - Grow vertical window split by 1
• ⌃w< - Shrink vertical window split by 1
• :(resize|res) (+|-)<n> - Horizontal resize to n or increase/decrease by n
• :vertical resize (+|-)<n> - Vertical resize to n or increase/decrease by n

Tabs

• gt - GoTo next tab
• g⇧t - Goto previous tab
• <n>gt - Goto tab n
• :tabedit <file> - Edit specified file in a new tab
• :tabfind <file> - Open a new tab with filename given
• :tabclose - Close tab
• :tabclose <n> - Close tab n
• :tabonly - Close all other tabs

coc.nvim

• ,dj Open implementation at cursor (coc-implementation)
• ,dd Open symbol definition at cursor (coc-definition)
• ,dr Open panel of symbol references at cursor (coc-references)

Denite

• ; - fuzzy-search buffers for a string
• ,t - fuzzy-search current directory for string
• ,j - fuzzy-search current directory for string at cursor

Help

• :h - Open Help
• ⌃] - Activate link at cursor
• ⌃T - Back

Search / Replace

• /<pattern> - Search forward for pattern
• ?<pattern> - Search backward for pattern
• n - Jump to next match in searching direction
• ⇧n - Jump to next match in opposite direction
• :<range>s/<foo>/<bar>/[g,i,I,c] - Replace old with new. range is either empty, % for whole file, or a line number. g flag replaces all occurrances on the line. i ignores case. I doesn’t ignore case. c asks for confirmation.
• * <n|⇧n> - Search forward for word at cursor, n jumps to Next result, N jumps to previous
• # <n|⇧n> - Search backward for word at cursor, n jumps to Next result, N jumps to previous

Find Files

For finding patterns in files. Use :vimgrep if unsure if the grep command-line utility is available, but :grep is generally faster. All searches fill a buffer that is accessible by the :cw command.

• :vimgrep /<pattern>/[g][j] <path/*/> - Search for regex pattern located in path. The g option specifies that all matches for a search will be returned instead of just one per line, and the j option specifies that Vim will not jump to the first match automatically.
• :grep <pattern> <path> - Search for regex pattern located in path.

This paper documents the creation and testing of a game playing artificial intelligence (AI) agent program. The agent is designed to play a game of Connect Four by Milton-Bradley. The game is played by dropping pieces into a game board consisting of a grid of 6x7 slots. The object is to make a vertical, horizontal or diagonal line of four pieces before the opposing player does. The agent designed in the current study is able to play against a human opponent or against another AI agent.

Introduction

In this chapter the rules of the game Connect-Four are described, as well as the task environment. We also introduce a naming convention used throughout this text.

The Rules of the Game

Connect-Four is a game for two persons. Both players have 21 identical pieces. In the standard form of the game, one set of pieces is red and the other set is yellow. The game is played on a vertical, rectangular board consisting of 7 vertical columns of 6 squares each. If one piece is put in one of the columns, it will fall down to the lowest unoccupied square in the column. As soon as a column contains 6 pieces, no other piece can be put in the column. Putting a piece in one of the columns is called a move.

The players make their moves in turn. There are no rules stating that the player with, for instance, the red pieces should start. Since it is confusing to identify for each new game the color that started the game, we will assume that the sets of pieces are colored white and black instead of red and yellow. Like chess and checkers (and unlike go) it is assumed that the player playing the white pieces will make the first move.

Both Players will try to get four connected pieces, horizontally, vertically or diagonally. The first player who achieves one such group of four connected pieces wins the game. If all 42 pieces are played and no player has achieved this goal, the game is a draw.

Diagrams 1.1, 1.2 and 1.3 show positions in which White has won the game:

In the position of diagram 1.1, White has made a horizontal winning group, while his winning groups were respectively vertical and diagonal in the other two diagrams.

A possible drawn position is shown in the diagram 1.4:

Task Environment

The goal of this study was to create an agent to play the game Connect Four. The rules for the game were taken from the classic game of Connect Four. However, the environment and the agent program was created from scratch. On the surface the program functions as a simple Connect Four game playable between two human players.

In order to make an appropriate agent design, the task environment must first be identified and described. Table 1.1 is a PEAS description of this task environment. The task environment can be further categorized by several other factors, such as observability, number of agents, etc. These factors are described in Table 1.2.

Naming Convention

In order to be able to talk about a position, it is useful to give each square a name. We have chosen to use a convention as used by chess players. The 7 columns are labeled ‘a’ through ‘g’ while the rows are numbered 1 through 6. In this way the lowest square in the middle column is called d1.

It is now possible to make a list of the moves made during a game. For the game diagram 1.1 this could have been:

1. d1, d2
2. c1, d3
3. e1, b1
4. f1, White wins.

It is also easy to use the names of the squares to show where the winning group was created. In diagram 1.1 the winning group was on squares c1, d1, e1 and f1. Since the squares must lie on a straight line, it is enough to specify the two endpoints of the group. In this case the group can be identified with c1-f1. In general the notation square1 square2 will be used to identify all squares on the line with sqaure1 and square2 as endpoints.

Complexity Analysis

In this chapter we show why a brute force approach will not be successful at this time.

Complexity of the Game

In order to get an idea about the complexity of the game an estimate is presented of the number of different positions that can be achieved, if the game is played according to the rules. A position that can occur during a game is called a legal position, while a position that cannot be achieved is called illegal.

Each square can be in one of three states: empty, white or black. Therefore it is easy to see that the number of possible positions is at most 342 (≥ 1020). This upper bound is a very crude one, and can be brought into better proportions.

If the total number of occupied squares in a given position is odd, the number of white pieces is one more than the numbers of black pieces. If the total of occupied squares is even, these numbers are equal. Furthermore, if a column contains an empty square, all squares higher than this square are also empty. If a position contains four connected pieces, the position concludes a game. Since the last move ended the game, at least one of the four squares in the connected group must be the highest filled square in its column. If this is not the case, or both players have connected four pieces, the position is illegal. If one player has more than one connected group this position can only be legal if these groups share a square that contains the last piece played. In the calculations we are going to make, we do not rule out positions in which are illegal for the reasons mentioned above. We also do not rule out positions that are not legal, because they cannot be achieved, during normal play. Diagram 2.1 shows such a position.

Although the position looks perfectly normal, it is clear that Black has made the first move. Therefore it is not legal simply because white is supposed to move first according to the rules.

We have calculated the number of different positions, including all illegal positions which contain too many connected groups of four pieces, and illegal positions as shown in diagram 2.1. For the standard 7 x 6 board, an upper bound of 7.1 x 1013 is found.

To determine the amount of memory needed to construct a database for Connect-Four this upper bound is useful. In order to show that such a construction takes too much memory, we need a lower bound instead of an upper bound. If we want to find a good lower bound of the number of possible positions, we have to make sure that each position we count is legal. Therefore all positions which cannot be achieved during normal play, e.g. diagram 2.1, should be ruled out. Diagram 2.2 illustrates the difficulties we are faced with in determining if a position is legal.

The position of diagram 2.2 is a draw. Although at first sight it might look like a normal position, it cannot be achieved during normal play. This can be seen as follows: the first move White made must have been d1. If Black played as his first move one of b1, d2 and f1, there is no possible second move for White. Therefore Black’s second move was one of a1, c1, e1 and g1. Suppose Black played a1, White then must have played a2 as second move, giving the position of diagram 2.3:

Now Black still cannot have played b1, d2 or f1, for the same reason as before. The move on a3 is not possible either. Therefore Black must have played one of the remaining c1, e1 or g1. After one of these, and after White’s answer to it, the position did not get any better. The furthest we can get with this game is shown in diagram 2.4.

In this position Black has to move. For all seven columns, the lower two squares should be filled by black pieces. Therefore after the next move of Black there is no move White can make that will eventually result in the position shown in diagram 2.2. Therefore that position is illegal.

This diagram shows that it can be rather difficult to detect if a position is illegal. It is equally difficult to show which of the positions are not legal because more than one group of four connected pieces is present. We therefore assume that a database should contain a large number of illegal positions. We believe that in that case the order of magnitude of the upper bound presented before, is a good estimate for the magnitude of the database. This number is by far too big to think seriously about making a database for Connect-Four. To see this, we have to consider the number of positions that must be stored at the same time when we build the database. When a retrograde analysis is applied, as has been done for many endgames in chess, we need not necessarily store the positions consisting of, say, 20 pieces, as long as we have not yet determined the value of all positions of 21 pieces. When we have determined the value of these positions, we no longer need the positions consisting of 22 pieces or more. Therefore we only need to be able to store all positions of n and n+1 pieces at the same time. For the 7 x 6 board, this means that we must be able to hold all positions of 36 and 37 pieces at the same time, a total over 1.6 x 1013 positions. We can store the value of a position in 2 bits, since we have 4 possible states: win for White, win for Black, draw or not checked (we can use the address of the 2 bits as identification for the position). This way we need at least 4 Terabyte. Therefore making a database does not yet seem realistic.

Development

Interface

The interface of the game program consists of two parts, the menu, and the game board. The menu gives the user access to the game controls, such as setting which AI algorithm to use, and which players will be controlled by humans and which by AI agents. One interesting feature of this particular design is that one can begin a game with two human players, and set up the board any way they like. Then one or both of the players can be switched to AI agents, in order to see how they react to different initial configurations of the board.

Once the players are set, the user can begin a new game. The game board is set to the initial configuration of all empty squares. Players then take turns dropping pieces into the 7 columns of the game board. Human players make moves by clicking the button above the desired row. This design was favored over other interfaces, such as clicking directly on the game board, partially because of time restraints, and partially because it more closely approximates the way the real game is played, by dropping pieces into the top of the board. The AI agents, of course, automatically make their moves, so their pieces appear very soon after the opponent makes their move. Since the AI agents are very fast at making their decisions, AI vs. AI games of Connect Four are for the most part very short.

When the game is over, a window pops up that notifies the user of the outcome. If the game results in a win for some player, the message states that that player has won. However, if the game ends in a draw, then this message is relayed to the user instead. From that point, the user can either quit or begin a new game.

As a side note, in early versions of the program, there was one other form of interface present. The list of moves in sequential order was output to a text window, along with system messages such as which player was victorious. This list of moves proved very valuable for the debugging process. At one point, when pitting AI agents against each other, the program became stuck in an infinite loop. It was very beneficial to be able to see what moves led up to the loop and what moves the AI was trying to make during the loop.

Design

This design is both intuitive and minimal. The GUI was written in Java/Swing, a toolkit provided by Java, as seen below in diagram 3.1.

Artificial Intelligence

The computer AI opponent of the program is configurable to different difficulty levels. Each difficulty level represents a different algorithm. There are four different difficulty levels in this version of the program; Random (easy), Defensive, Aggressive, and Minimax (difficult).

Random AI

The Random AI algorithm simply places pieces randomly each turn. This algorithm can be defeated easily by human players and by the other AI algorithms. It is also the only one that is non-deterministic. Since it randomly places pieces, the move progression will be different each time this algorithm plays. As described below, the Random algorithm is the only one with this characteristic.

Defensive AI

The defensive AI algorithm uses a heuristic function to determine what the next move should be. It looks at the current state of the board and assigns a value to each of the available moves. The higher this value is, the more dangerous it is not to move there. For instance, if the opponent has three pieces in a row, a value of 8 is given to the space that would complete the opponent’s four-in-a-row. If the opponent has two pieces in a row, a value of four might be given to the adjacent square. The point of this algorithm is to block the opponent from getting four in a row at all costs.

Aggressive AI

The Aggressive AI algorithm uses the same type of heuristic function used by the defensive algorithm, with one key difference. The defensive algorithm only applies the heuristic function to the opponent’s pieces on the board. The aggressive algorithm applies it to both the opponent’s pieces and its own pieces. Thus it simultaneously defends against potentially losing situations by blocking the opponent from winning, and makes offensive moves maximizing the number of pieces it has in a row. Skilled human players are still able to defeat this algorithm fairly easily, however, since it does not look very far ahead in the game tree.

Minimax AI

This algorithm was initially going to use the minimax algorithm (as implemented in Artificial Intelligence: A Modern Approach) to search the game tree for the optimal move. However, the tree proved to be too massive to search in this way, due to computational limitations. One possible solution to this problem was the alpha-beta pruning algorithm, in which game subtrees are eliminated to cut down on the number of computations. However, this idea was passed over for a simplified version of the minimax algorithm. Instead of searching the entire game tree, the algorithm used in the final version of the program only searches the tree up to a certain point. So, in effect, the algorithm is looking four moves ahead and making the best move based on that knowledge. The decision to limit the lookahead to four levels was made because it is about equal to the number of moves a skilled player can look ahead, and it is a good match for the current power of processing technology. Thus the Minimax AI is a good match for a skilled human player.

Results

One of the primary functions of this study was to learn about how the different AI algorithms perform against one another. Due to the short length of the matches played between to AI players, many trials were able to be conducted. Table 5.1 is a brief summary of how the different algorithms interact.

Conclusion

Above all else, this study was designed so that the authors could learn more about AI. This goal was successfully accomplished. The specific area in which the most experience was gained is that of computer game playing agent design. While the algorithm best suited to success may be impossible to execute with current technology, other algorithms are just as suitable to make for an interesting playing experience. This is an important game design concept, because more often than not, people want to win games they play. Therefore, if the computer AI always wins because the algorithm is perfect, the game will suffer from lack of human interest.

Heuristic algorithms like the Defensive and Aggressive algorithms outlined in this study are very well suited to solve this problem. Since they do not always win, but sometimes trick a human player, they retain a human player’s interest. As long as no one programs a heuristic algorithm to become frustrated at losing, they will remain one of the best solutions to a computer game-playing problem.

References

The purpose of this paper is to provide an overview of the existing literature concerning speech segmentation, categorical perception, and some other issues concerning bilinguals of English and Japanese. It will touch on some of the differences in the two languages and how they affect learning the L2. The paper will start by providing background information about some of the two languages and some of the current issues involved in speech processing. The paper will then delve into the most current research, what the issues are, how it was done, and the results they have found. It will then go on to discuss the possible future directions of this research and end with references.

I. Introduction

People trying to learn a second language always have a difficult task ahead of them. Learning a new grammar and lexicon takes both time and practice. Some languages are diverse enough that on top of a new lexicon and grammar, there is also an entirely new segmentation system to be learned. Japanese and English are diverse in the way the speech streams are segmented into parts, with Japanese using the mora as the basic unit of perception (McQueen, Otake & Cutler, 2001) and English using stress (Cutler & Butterfield, 1992). Along with learning a new lexicon, grammar, and segmentation system is the problems with categorical perception. How can bilinguals learn new sounds that aren’t in their native language? Another important issue is if there are multiple segmentation cues, can there be universal segmentation cues besides the rhythm-based mora, syllable and stress-based processes? This paper will try to uncover these questions.

Background

When participating in conversation the listener must do a myriad of tasks to comprehend what the other speaker is saying. On the physical side of things, the comprehended must process the sound waves from the air into electrical signals into the brain. From there he/she must then begin the process of turning sound signals into words, phrases, sentences, and finally into a complete dialog. The aspect of changing the raw speech stream into words is called speech segmentation (Cutler, Hehler, Norris & Segui, 1986). Speech segmentation is usually accomplished in different ways according to the language, the major rhythm-based segmentation types are mora in Japanese, syllable in French (Cutler, Hehler, Norris & Segui, 1986), and stress in English. The mora is the smallest Japanese unit of perception, it is subsyllabic. “It can be a vocalic nucleus, a nucleus plus syllable onset, or, as in the second and fourth morae of shinshinto, it can be the postvocalic portion of a syllable, i.e., the coda.” (Otake, Hatano, Cutler & Mehler, 1993). Japanese is a “mora-timed” language, where each mora represents a rhythmic unit; in comparison to English which is “stress-timed” (Beckman, 1982). There has been much research into the field of speech segmentation, specifically into the types listed above including stressed-base and syllable-based (Cutler, Mehler, Norris & Segui, 1986), as well as mora-based (Cutler & Otake, 1994). There has also been a fair amount of study on phoneme discrimination of /r/ and /l/ in native speakers of Japanese and English. One particular study (Miyawaki, Strange & Verbrugge, Liberman, Jenkins & Fujimura, 1975) used synthesized speech to compare Americans and Japanese at discriminating /ra/ and /la/. A follow-up study a few years later reinforced previous findings that Americans can categorically discriminate the phonemes, and that Japanese are at a near-chance level of perception (Strange & Dittmann, 1984). This study also showed that after training, the native Japanese performance on those phones increased, and therefore categorical perception is indeed possible for non-native speakers.

II. Review of the existing literature

1. Discrimination of /r/ and /l/ by native speakers of Japanese and English

Categorical perception of speech sounds is an important aspect of speech segmentation. Without the ability to properly hear differences in non-native speech sounds, then segmentation and comprehension will be negatively impaired. One of the seminal research papers written comparing the abilities of English and Japanese natives to discriminate the /r/ and /l/ sound was by Miyawaki, Strange, Verbrugge, Liberman, Jenkins and Fujimura in 1975. The research here confirms earlier findings that Japanese subjects cannot distinguish between /r/ and /l/ (Goto, 1971). This paper investigated the effects of linguistic use on the ability to discriminate the class of liquid phones in English and Japanese natives. They focused on the phonemes /l/ and /r/ in syllable-initial position. The choice for the liquid /l/ and /r/ phones were made due to the fact that “the distinction between these phones is phonemic in English but not in Japnanese.” (Miyawaki et al., 1975). The /r/ and /l/ phones don’t constitute a phonemic contrast in Japanese, and therefore would provide a good base to conduct tests on the differences in native and non-native discrimination tasks. This research paper used a speech sound generator to create a series of sounds between two phones in order to see where and how categorical perception would occur. The parallel-resonance synthesizer generated 15 3-formant sounds that would be used in the tests. The third formant (F3) was varied in frequency in steps between the /ra/ and /la/ sounds. From this set of 15 sounds, there would be two types of tests conducted on each subject group, an identification test and an oddity discrimination test. The subjects of this research consisted of 39 native American speakers and 21 native Japanese speakers. The discrimination task was to listen to a series of three sounds, and make note of which of the three was different. The results showed that the Americans could easily discriminate the target sound, only getting low scores if the sounds were ambiguous as to if it was /r/ or /l/, “pairs whose members were labeled as the same phoneme was considerable less accurate, although still above the 33% chance level” (Miyawaki, et al. 1975). The Japanese however, showed a near-chance level of discrimination. An interesting aspect of this study was the finding that when speech sound context was not included, the two groups behaved almost identically, “we see very clearly that the Japanese do not differ from the Americans on any of the comparison pairs. The nonspeech discrimination functions are virtually identical for the two groups of subjects.” (Miyawaki, et at., 1975). Both groups were able to discriminate isolated F3 patterns quite accurately, which indicates that both groups are able to hear sound differences physical sub-contextual level.

A later study on the /r/ and /l/ perception task using a synthetic speech generation process similar to the above study focused on if linguistic experience had an impact on categorical perception of the /r/ and /l/ phonemes. They were suggesting that “… native Japanese adults learning English as a second language are capable of categorical perception of /r/ and l/l.” (MacKain, et al., 1981). This study was similar to the above, but had a few differences. The first was how they varied the acoustic values both temporally and spectrally “to optimize the Japanese subjects’ opportunity to show perceptual differentiation of the /r/-/l/ contrast.” (MacKain, et al., 1981). The second was the inclusion of an AXB oddity discrimination task which is thought to provide a better opportunity to let the subjects detect auditory differences. The third was an identification task for both the American and Japanese groups. The Americans, as expected, displayed a strong category boundary with strong identification scores. The not-experienced Japanese subjects displayed poor categorization of the /r/ and /l/ with near chance levels in all stimuli types. The results in the non-experienced Japanese group extended the Miyawaki results. The experienced group “had intensive English conversation training with native American-English speakers and as a group spent a larger percentage of the average day conversing in English than did the not-experienced Japanese.” (MacKain, et at., 1981). This group displayed similar results to the Americans on both the identification tasks and the discrimination tasks. These results suggests “that the occurrence and abruptness of an /r/-/l/ category boundary for the experience Japanese might be related to their grater conversational English experience…” (MacKain et al., 1981). This particular study had added the AXB oddity task because it is less memory demanding and “they could use nonphoenetic auditory memory to aid performance” (MacKain et al., 1981) in the hopes that it would allow the non-experienced Japanese to obtain better results, but this particular task did not achieve the hopes of its design intentions. Overall, their research suggested that Japanese native speakers can obtain categorical perception of /r/ and /l/ with some practice and more experience, which is good news to aspiring bilinguals.

Research into the /r/ and /l/ phonemes is interesting particularly to Japanese because of the lack of contrast between these sounds. Distinctive contrast in particular speech sounds allows speakers of that language to discriminate where other language speakers would not. The above studies examined this aspect as well, but in a 1984 study by Strange and Dittmann focused on how the ability to discriminate sounds not available in the L1 can change with explicit training, “We were interested in whether we could modify the perception of AE word-initial /r/ and /l/ by adult Japanese learners of English, in the laboratory, using the psychophysical training task successfully employed by Carney et al. (1977).” (Strange & Dittmann, 1984). The design of this study was to test the abilities of eight female Japanese native speakers before training and then again after training to examine the effects of training on discrimination of the /r/ and /l/ series of synthetic speech sounds. The initial pre-training tests consisted of a minimal pairs test, an identification of the rock-lock series and an oddity discrimination task. The training was done individually over a three week period that totaled 14 to 18 sessions, it consisted of an AX discrimination task with immediate feedback. At the end of the training the post-training tests were given, which were the same as the pre-training tests. Pre-training results were similar to the results found in by MacKain in 1981, with near-chance levels of accuracy. The training task performance showed “gradual improvement over session with the greatest improvement in the first several sessions” (Strange & Dittmann, 1984). All subjects showed increased performance as the training sessions progressed. After the training, the post-training tests showed that “pretraining versus posttraining categorical perception tests for each of the eight subjects of the rock-lock series revealed that seven of the eight subjects improved as a function of the training.” (Strange & Dittmann, 1984). Post-training test results also showed improvement in their discrimination of cross-category pairs with over 75% correct (Strange & Dittmann, 1984). Overall, training did indicate improvement as performance on post-training tests showed better performance results (Strange & Dittmann, 1984). “We can thus conclude that training with a fixed-standard AX discrimination task resulted in improved (categorical) perception of the training stimuli, as tested by the (more demanding) identification and oddity discrimination tasks” (Strange & Dittmann, 1984). This study also tested differences in sound according to acoustic properties. They wanted to test if training in the /rock/ - /lock/ set would transfer over to good performance in a similar /rake/ - /lake/ test. Their results showed that it was indeed the case, and training did help this test achieve improved performance, this also supports the idea that bilinguals can indeed learn to perceive new sounds in a second language.

2. Speech segmentation using the Mora

One of the major differences in a language is how it is timed, or the rhythm of the language. Bilinguals not only have to learn a new lexicon and grammar system, but sometimes must also learn a new rhythm to speak their L2. An example of this would be a native English speaker trying to learn Japanese or vise-versa. English is a stress-timed language whereas Japanese is mora-timed. Research of the mora can help lead to further understanding of how languages of different timing style can be learned more efficiently for bilinguals as well as how differences in timing can affect different aspects of speech segmentation.

In a study by Otake et al., 1993, of how Japanese words are segmented by native and non-native listeners, results found that Japanese responses were consistent with moraic segmentation, while non-native listeners responded differently. The mora is a uniquely Japanese timing mechanism; it is smaller than a syllable, and is considered the basic unit of perception. This study set up four experiments with native and non-native speakers to examine the effects of native timing type on Japanese words. In the first experiment 40 native Japanese speakers were used to listen to a series of 3 to 6 words. When a word was heard with the target letters on a printed card, the subject was to press a button. Results showed that the native Japanese listeners responses better supported the mora hypothesis than the syllable hypotheses, confirming their initial hypothesis that Japanese speakers will segment words best with the mora-timed segmentation style. “The pattern of results in this experiment thus appears to offer strong support for the mora hypothesis but none to the syllable hypothesis.” (Otake et al, 1993). The second experiment tested English speaking subjects on Japanese words. The design was similar to the previous task on the Japanese subjects, where a series of Japanese words would be played, and the subject was to press a button once the sound on the printed card was heard. The results of this experiment are in stark contrast with that of experiment 1. The findings in this experiment support that “we many now conclude that English listeners do not exploit mora structure in the same way.” (Otake et al., 1993) [as Japanese native listeners]. A note about experiment 1, the targets were presented on printed card in Roman text orthography, this can introduce confounds due to how Japanese naturally represent sounds with kana characters. Experiment 3 was exactly the same design as experiment 1 with the exception that the target word was played through the headphones before the word sequence rather than being printed on a card. Experiment 3 had 40 native Japanese speaking subjects. The target sound was played first, followed by the sequences of test words. The subject would press a button when they heard the target word in the sequence. The results of experiment 3 replicated those of experiment 1. “The replication of experiment 1’s results strongly confirms our conclusions from the preceding experiments: Japanese listeners do not naturally segment speech syllable by syllable; they do naturally segment it mora by mora.” (Otake et al., 1993). These findings also cleared up the problem of orthography of Japanese speech sounds mentioned earlier. Experiment 4 was identical to experiment 3 with the exception of the subjects being 33 native French speakers. The results of experiment 4 were as expected, “the response patterns of French listeners are, as predicted, best accounted for by the syllabic hypothesis…” (Otake et al., 1993). These results also support findings by Cutler et al., 1986. Finally, these results support the predictions made initially, that non-native listeners will “not replicate the pattern of results shown with the same materials by native Japanese listeners” (Otake et al., 1993). These results support that non-native listeners will try to segment the sounds of a non-native language by applying their native speech segmentation system, whether its mora-based, stress-based or syllable-based. How could that affect learning a second language where the segmentation process is different from the native language?

Taking from some of the findings from the previous study, Cutler and Otake started a new research project in 1994 that would focus on whether subjects would apply their native segmentation processes to a foreign language. This research could show that inappropriate use of a segmentation process could inhibit the processing of a non-native language in bilinguals. “This suggests that segmentation procedures may indeed be highly similar to phonological categorization procedures: they effectively aid processing of the native language, but they may reduce processing efficiency for input in a foreign language.” (Cutler & Otake, 1994). This study also hoped to further support the mora hypothesis put forth by Otake in 1993. The first experiment was to test Japanese native speakers on if moraic targets “will be easier to detect than nonmoraic” (Cutler & Otake, 1994). This should prove to be the case under the mora hypothesis. This experiment will also test for whether phoneme detection “will be differentially difficult for vowels versus consonants” (Cutler & Otake, 1994). Experiment 1 was designed with 40 native Japanese speakers, they would listen for a target sound (O or N) in a series of Japanese words, and press a response key as soon as they detected the sound. The results from this experiment showed the same results seen in Otake (1993): “mora structure is crucially involved in the process by which Japanese listeners convert spoken input in lexically accessible representation.” (Cutler & Otake, 1993), as well as showing that quick responses to moraic input is not only restricted to CV input as seen in Otakes’ 1993 experiment. It should be noted that there were no significant findings concerning the differences in vowel and consonant response times mentioned earlier. Experiment 2 was conducted in the same way as experiment 1, with the exception that the subjects were 24 English native speakers. As expected, the non-native speakers did not demonstrate the moraic effects shown in experiment 1. Interestingly, “The main effect of vowel versus consonant target was, however, significant; consonants were detected both faster and more accurately than vowels” (Cutler & Otake, 1994). This supports the earlier findings that native listeners apply their native segmentation procedures to a non-native language. Experiment 3 was similar to experiment 2 except with English words as both target and sequence words. The subjects were 24 native English speakers. Findings in experiment 3 showed that “These findings are in line with previous failures to find significant phoneme detection differences between targets in stressed versus unstressed position with English listeners and laboratory-read speech.” (Cutler & Otake, 1994). These results were similar to experiment 2. Experiment 4 used the same materials as experiment 3, English words, while it used the same design, as well as the same subjects as in experiment 1, using Japanese native speakers. Results in this experiment found the vowel sounds were much more difficult to find than the consonantal targets (Cutler & Otake, 1994). The findings of experiment 4 also help support the mora hypothesis, namely “that mismatch between in the input and the native language phonemic repertoire plays a role in phoneme detection in a foreign language.” (Cutler & Otake, 1994). Experiment 5 was designed with 20 native Japanese speakers listening to a played back words of native Japanese. The procedure was that of experiment 1. The subjects were to press a button when they heard a target phoneme in the played back word sequence. The results of this experiment demonstrated what the experimenters had hoped for, “a new effect in Japanese phoneme-monitoring: targets in word-initial position are detected faster than targets in word-medial position.” (Cutler & Otake, 1994). This experiment showed the mora effect was significant for all four phoneme targets. Experiment 6 involved doing the same as experiment 5 but with English subjects. This experiment was designed to show the English speakers advantage of detection of vowels over consonant targets, as well as observing a difference in RTs in A and O due to “phoneme repertoire mismatch” (Cutler & Otake, 1994). This experiment used the same materials in experiment 5 with 23 native English speakers using the procedures from experiment 2. The results of this experiment showed that “there was again a significant overall advantage for consonant over vowel targets” (Cutler & Otake, 1994). Overall these experiments had shown that the “the moraic effect which Japanese listeners show in phoneme detection in their native language appears in world-initial well as in word-medial position and with a variety of phoneme targets.” (Cutler & Otake, 1994). These experiments also go to explain that the consonant and vowel detection abilities of English speakers appear when listening to Japanese, due to native English segmentation process advantages in this particular task.

In terms of bilinguals, the above studies used Japanese native speakers, some of whom had some English experience. Even with experience, it was shown that they were still applying mora segmentation to English input, as well as English speakers applying stress-based segmentation when it is inappropriate in Japanese input. As quoted from the above study “We believe that this finding has potentially important implication for understanding the processes of acquisition of a second language.” (Cutler & Otake, 1994). Although this research seems to cast a dim picture on those trying to learn a second language, all is not lost, in a study by Cutler et al., 1992, showed that more than one segmentation process is available, and even if not available, these rhythm based processes are heuristics for processing a non-native language, are not necessary for comprehension. (Cutler et al., 1992).

3. Rhythmic cues and the Lexicon

An important area of study is on how rhythmic cues in the speech stream affect lexical decisions in Japanese. A learner of Japanese would be affected by this in that many languages don’t use rhythmic cues, and so in learning Japanese, rhythmic cues may not serve as an available heuristic. Japanese provides a good test platform to study rhythmic cues and speech segmentation because Japanese “rhythm is based neither on syllables nor on stress. Instead, it is based on the mora, a subsyllabic unit which can be of five different types” (McQueen et al, 2001).

In a study by Cutler and Otake, examining old Japanese wood-block prints which contained a word-based joke system called “Goroawase” to examine if there existed sub-moraic information processing in the speech stream. This joke system is used by substituting a single mora in a word with another mora to create a similar sounding word with a different meaning, thus creating a word-pun. Their findings suggest that “mora substitution is more often than would be expected by chance in effect phoneme substitution because two words which overlap in all but a single consonant or vowel form a better target-pun pair than two words which overlap in all but a CV mora.” (Cutler & Otake, 2002). Another experiment in the same paper uses word reconstruction. They had subjects listen to a word with a replaced mora, and were told which mora was changed. The subjects were then to tell which word had been intended. This experiment would use mora as the cue to the target word, which would then be accessed via the lexicon. 45 native Japanese speaking subjects partook in this experiment and the results were as expected. 4 mora words were more easily accessed than 3 mora words. Their results suggested that “word reconstruction was significantly easier when the initial mora had been replaced by another mora sharing with it either C or V.” (Cutler & Otake, 2002). In their second experiment, they distorted the final mora of the word, and found that identification was faster and more accurate, which suggests “that this information can be exploited continuously rather than only on a mora-by-mora basis” (Cutler & Otake, 2002). This indicates that lexical access can occur without complete moraic information. In experiments 3 and 4 of their research, they tested replacing both third and fourth mora and placed a focus on consonant versus vowel replacement, they found similar results to experiment two. “Although both the V-replacement and C-replacement condition proved easier than M-replacements, there was also a difference between the first two: Replacement of a vowel proved easier than replacement of a consonant.” (Cutler & Otake, 2002). On their 5th experiment, they used a Yes/No type response to the design of the previous experiment, their results helped to cement their conclusion that there is continuous speech processing, even below the level of the mora. (Cutler & Otake, 2002). This does not suggest that there is no mora, but only that continuous speech sounds are helpful to the segmentation of the continuous speech stream (Cutler & Otake, 2002). Their results do however suggest, and support the findings of Norris et al., in 1997, “that the contribution of rhythmic categories in word recognition is the same for all languages” (Cutler & Otake, 2002). As rhythmic categories are universal to all languages (Norris et al., 1997), this would provide some relief to learners of Japanese a second language, as mora are not the end-all to segmentation, but a heuristic function that helps to make segmentation more efficient.

If mora and meter are both parts of speech segmentation, how can we combine the two? A study by McQueen, Otake and Cutler in 2001 tries to answer that. Their experiments use the PWC, or Possible Word Constraint developed by Norris et al., in 1997 to test if Japanese speakers use the PWC just as English and Dutch speakers. The PWC is another type of heuristic function used in speech segmentation particular to the Shortlist model developed by Norris et al., (1997). Their first experiment used 54 native Japanese speakers to listen to a native speaker of Japanese pronounce nonwords with Japanese words embedded within them. The subjects were then to press a button and pronounce the embedded word once they heard it. The results of the experiment showed that indeed, “listeners find it harder to spot words in impossible word contexts than in possible word contexts. Japanese listeners therefore appear to use the PWC when segmenting speech.” (McQueen, Otake & Cutler, 2001). Other tests done in this study all supported the PWC and show similar results done with English speakers. (McQueen, Otake & Cutler, 2001). The PWC suggests the speech segmentation is more universal than previously devised models. This model suggests that words are activated and no particular lexical decision is 100% decided, until all are decided. “The present experiments therefore support the theory of lexical segmentation that the PWC offers. On this view, candidate words are activated by the incoming speech stream and compete with each other until a lexical parse is settled upon.” (McQueen, Otake & Cutler, 2001). This model also allows for the rhythmic segmentation process to exist and help add to the parsing of lexical and segmental structure. “Furthermore, just as the rhythmic structure of English or Dutch provides English and Dutch listeners with cues to the location of likely word boundaries (Cutler & Norris, 1988, Vroomen et at., 1996), so too does the characteristic rhythm of Japanese provide Japanese listeners with a segmentation cue.” (McQueen, Otake & Cutler, 2001).

Rhythmic segmentation provides another type of segmentation heuristic along with mora-based segmentation as seen in the above papers. Although bilinguals have trouble when inappropriately applying their native language segmentation style to a non-native language, rhythmic segmentation is a universal heuristic which exists to compensate when L1 segmentation style fails to properly segment the non-native language.

4. Segmentation in non-native languages

An important area of study concerning bilingual speech segmentation is how non-native speakers segment their second language (L2). If non-native speakers can use any of the non-native language rhythmic, prosodic, lexico-syntatic, and syntactic segmentation processes, then they will have a much easier time segmenting the L2. The ability to learn a non-native segmentation process would be beneficial to any bilingual as it would facilitate comprehension of the L2. Research into brain plasticity for learning non-native language processes would directly benefit the existing body of knowledge.

In a study of brain plasticity and non-native lexical and segmentation processes, Sanders et al. in 2002 came up with interesting methods for probing non-native speakers to see if learning the L2 early in life or later in life would effect the ability to learn non-native lexical and segmentation processes. This study would research various language subsystems such as lexico-semantic, syntactic, and prosodic information processing. Using the earlier proposed theories that each language uses different methods of speech segmentation (English: Stress-based, French: Syllable-based, Japanese: Mora-based) they tried to find if non-native speakers used lexico-syntactic information processes when segmenting a non-native language (Sanders et al., 2002). Four groups were obtained to study this hypothesis, a group of native English speakers as a control group (E), early English Japanese native speakers (JE), early English Spanish native speakers (SE), late English Japanese native speakers (J), and a group of late English Spanish native speakers (S). “If non-native speakers fail to use rhythmic segmentation cues other than the rhythmic cues relevant to their L1, native speakers of Japanese (mora-timed) and Spanish (syllable-timed) would not be expected to use stress pattern as a segmentation cue when listening to English. Alternatively, native Japanese and native Spanish speakers might differ in their abilities to use stress pattern as a segmentation cue in English.” (Sanders et al., 2002). For this experiment they created 5 groups of 3 sentences each. The 5 groups were “Strong stress, initial position (SI), strong stress, medial position (SM), weak stress, initial position (WI), weak stress, medial position (WM), and target absent (TA).” (Sanders et al., 2002). Each group contained 3 sentences each of which were a semantic, syntactic or acoustic target sentence. The semantic words were normal English sentences, the syntactic sentences replaced all open-class words with non-words, and the acoustic words only retained the original prosody. Participants were asked to listen to a target, and then were asked to press one button if that target was heard in the beginning of the sentence, another button if it was heard in the middle, and the third button if it was not heard. Results indicated that “the fact that both groups of late-learners were able to use the lexical information supports the hypothesis that the lexico-semantic system remains relatively plastic beyond the age of 12.” (Sanders et al., 2002). These results support the idea that late learners can learn non-native language processes later in life, which is a reassuring fact for late-bilinguals, however “No group of non-native speakers used syntactic information to the same extent as native speakers.” (Sanders et al., 2002). These results, as well as other indicate that syntactic information processes are not as easily learned later in life (Sanders et al., 2002). An interesting finding was how both early and late learners of English Japanese-natives were able to use some segmentation cues that are seemingly effective in both English and Japanese; “this study could either indicate that both groups were applying a Japanese segmentation cue that happens to co-occur with stress in English or that both groups had enough exposure to English to learn a new segmentation cue.” (Sanders et al., 2002). Overall the findings of this study show that both lexical and semantic segmentation subsystems retain the “ability to change to a greater degree than do syntactic subsystems” (Sanders et al., 2002).

In terms of bilinguals, some very reassuring findings from the above study is that some segmentation processes are still learnable later in life, and they are also interchangeable, one or more can be used when another is unavailable or not applicable. “The findings also indicate that segmentation cues can be used flexibly by both native and non-native speakers, such that cues that are both available in the speech stream and usable by the listener are employed to a greater extent when other segmentation cues are either absent or not accessible to the listener.” (Sanders et al., 2002).

III. Possible directions for future research

In studies such as Otake (1993), findings were made that support the idea that non-native speakers segment a foreign language using their native segmentation processes. Future research could focus on bilinguals learning an L2 that has a totally different segmentation process. An example would be how a Japanese learning English or an American learning French begins to learn a new segmentation style, and how specific training on segmentation may make learning more efficient. A similar study design that could be used would be Strange & Dittmanns’ (1984) study on the /r/ - /l/ distinction of Japanese speaking English. This study could be applied to learning a new segmentation style. The time of acquisition of a new non-native speech segmentation process would also be of interest to this field. Also of interest to future research is the topics of multiple segmentation cues, such as lexico-semantic, and prosody, and how these can be used interchangeably along with more native segmentation processes such as mora, syllable, or stress-based segmentation.

Bibliography

Introduction

Hikikomori (Jap. 引きこもり) and otaku (Jap. おたく) are considered today’s most prominent social problems. Hikikomori refers to the recent state of middle and high school students who drop out of school, and withdraw completely from society. Otaku are the deeply obsessed fans of a particular subject who commit their free time (and sometimes lives) to the complete memorization of their obsession.

In both cases these groups are extreme exaggerations of common Japanese social tendencies. The media in Japan is currently attacking these groups as harbingers of social chaos, and portrays them as a new group of outcast who will destroy society (at least kill many people). This portrayal is not only accepted by the media, but by the society as well.

History has shown the Japanese social tendency to create outcast groups, isolate them, and then blame current social problems on that group. These groups are then used as examples of “The Other”, a group that is in opposition to the main Japanese society. This “Other” is used to construct a national and cultural identity by comparing it in opposition to the norm.

What defines a person as Japanese? If you asked a Japanese person today that same question, they would have an easy time answering that question with a quick “A Japanese person is a person from Japan.” This answer does not consider people of foreign heritage born in Japan, such as Koreans, or children born from migrant workers. If you want to be Japanese, you must look, speak, act, and most importantly be by-blood, Japanese.

In short, while it is possible-though not particularly easy-for a foreigner to acquire Japanese citizenship, it is not possible for an immigrant or the children of immigrants to “become Japanese” the way such people can “become American.” The way you get to be Japanese is the same way you get to be Zulu or Shona in Africa: you have to be born into the tribe. For that is what the people of Japan-or at least more than 97 percent of them-really are: members of a single great tribe united not just by common citizenship or common language but by common bloodlines, common racial memory and common tribal codes, so of which stretch back into prehistory.

Japanese society distinguishes itself from other cultures based on in-groups and out-groups. The groups that are not Japanese, not local, and different are considered the out-groups. Local groups, family groups, and traditional groups are considered Japanese. To define oneself as Japanese you must simply define yourself as a member of the in-group, and others as members of the out-group. This phenomenon has created an unconscious system of creating out-groups with each generation in order to achieve national self-identity.

The Past

Looking into the past, we find a long tradition of an outcast system that has continued right up until today’s modern society. With outcast groups such as the Hinin (Jap. 非人) class starting in the Heian period, which evolved into present-day Burakumin (Jap. 部落民), or Buraku. Later, Westerners were seen as the new group of outsiders, bearing prosperity, change and new evils.

Later still we find the in surge of Koreans to fill the immediate need for industrial labor.

In the last few decades, the outsiders haven’t been literally from the outside of Japan, or from outside of the caste system, but instead from inside the deeper parts of the society, the seemingly average citizen. These people are deemed socially inept, different, and therefore put to the outside.

The Buraku has existed since the 11th century, originally with such names as eta and Hinin. They first were considered members of a class outside of the traditional caste system of Feudal Japan. Today they characterize many of the same ways of life as they have in the past. Most of these people work in the so-called Buraku industries of shoe making, dyeing, slaughtering animals and handling their skins, providing laborers for crematory or graveyard work, and scrap collection of all kinds. Buraku also participate in unskilled low-level construction, garbage collection, street cleaning, as well as leather working.

Beginning with the Spanish and Portuguese seafarers in the mid-sixteenth century, westerners were easily identified as the largest group of strangers, or outsiders to the Japanese. While these people were easily identified as outsiders because of their foreign origins, they were also outsiders because of the way they were seen and accepted by society.

At the same time however, outsiders themselves were ambiguous figures, for usually they were believed to possess both beneficent and malevolent powers. The stranger our outsider might have skills pleasing or valuable to the community and be the harbinger of good fortune in general, but he or she could equally well be the bringer of calamity, the bearer of evil.

Koreans have been a largely disenfranchised group of Japanese society since Korea was colonized in 1910. The situation of Koreans in Japan is that many were brought into the nation as migrant unskilled laborers for the immediate needs of the industrial revolution. The problem arises when considering the consequences of a large group of foreign people who are thought of as never being able to assimilate into a homogeneous society.

Japan today doesn’t allow Japanese-born Koreans to become citizens despite the fact that their families have lived in Japan for up to three generations. The Korean population is largely located in Osaka. There is currently a movement to promote full citizenship to all resident Koreans, and equal status as a Japanese citizen.

The Present

Koreans and Buraku are still considered out-groups in Japan today, however, they aren’t nearly as publicly discriminated against as the more recent groups that have been so popular in the media. Recently there has been a surge of discrimination against groups of Japanese peoples within Japanese society. These groups consist of socially divergent individuals who sometimes identify themselves as members of a specific group. Such groups are the religious cults, otaku and most recently the hikikomori.

The cult phenomenon in Japan has ties with the traditional Japanese connection with self-identification with a specific group. These values go back to the small villages that primarily grew crops, and survived as a group by trading produce and services within the small-encapsulated society. These villages would be self-sufficient and reluctant to welcome strangers into the group. The cult phenomenon is similar in many ways. In the last fifteen years, cults have grown in size as well as numbers, the most prominent being the Aum Shinrikyo (Jap. オ ウム真理教). These cults are isolated from the rest of society, its’ members are not allowed to communicate with the outside world. With the acts of terrorism in 1995 by this cult, cults are now seen as outsiders by the majority of Japanese society.

Otakuおたく have arisen out of the early 1980s as a new group of mostly teenage and twenty-something year olds who share a common obsession. The term Otaku is derived from its ancient meaning of addressing someone from another household, and implies that you are unconcerned with the details of the other persons family and circumstances. It prevented the discovery that there were differences between ‘I’ and ‘You’.

Today the term is used to mean

I am not interested in you or your inner life. I share with you only a narrow interest.

Usually the obsession is about an obscure topic such as a dead rock star, a video game, an animated (anime) cartoon series, computers or internet. These groups are seen as strange people who try to learn everything they can about an obscure topic, and therefore are different, and should be avoided.

The Hikikomori are the most recent group of deviants whom are shown in the media as the most troubled, and enigmatic. Similar to otaku in certain ways, these people frequently have an obsession, but not always. Loosely translated as “social withdrawal”, it refers to the state of anomie in which an increasing number of young Japanese seem to fall into. These people typically refuse to go to school (Jap. 不登校), lock themselves in their bedrooms, and refuse outside contact with society. This group is currently being scapegoated for a variety of social issues from school stabbings, and general violence.

The Victims

Otaku can appear to be anyone, they could be your standard citizen, however with a few peculiarities. To be an otaku means much more than to have an extreme obsession or passion. Weather his/her specific knowledge pertains to a particular field, such an anime, manga, networking computers, realistic military models, or even daytime soap opera; or a more specific field such as a long-run Japanese anime series such as “Neon Genesis Evangelion”, or the character traits of every pokemon monster; all otaku strive for the same thing which is to fully immerse themselves into every minute detail of their object of obsession.

Although otaku frequently identify themselves with people with the same obsession, this is merely a superficial connection. Most otaku never actually meet each other, but instead communicate through the Internet. Otaku acknowledge the inability or futility to achieve the intimacy of ‘authentic’ human interaction. They draw a protective barrier around themselves, and use their obsession as the bricks in the wall. Communication between otaku of the same obsession at it peak would be chatting, face-lessly over the internet, safe behind their monitors in their own homes.

Otaku spend their time obsessively memorizing and analyzing facts. If they love computers, then they will read everything and anything about computers they can find, from how to install a strange operating system-to designing their own robots. If they obsess over the latest daytime drama, then they will memorize the characters lines, and perhaps write stories involving the characters in different situations. This act in itself is an act of perseverance and conformity to the education system that he/she was brought up in. Both of these qualities are seen as having ‘virtue’ in Japanese society, but when brought to extremes can have serious social implications.

This is the purest imaginable display of the narcissism inherent in Japanese society. The otaku desires both an idealized union and an impregnable independence-the classic drives of the narcissist. He seems postmodern and marginal but is deeply traditional in his rejection of the unfamiliar. Students in the ancient schools of tea ceremony were like otaku. Every member had to be a mirror of every other member. The otaku represent rebellion as parodic conformity.

The media have recently jumped on the rash of violent activities of a few otaku and have made the group into monsters. The rate of teenage violent acts of crime has not increased dramatically in post-bubble years, however the media now has a scapegoat to points its finger at. Now whenever a violent crime occurs, the media jumps to cover the latest details, and point the finger at violent otaku as the culprits. One such incident was the seventeen-year-old boy who bludgeoned passengers at a Tokyo subway station with a baseball bat. A very well known incident was the Kobe murders of a child whose head was placed on the wall of the schoolyard. The teenage culprit was deemed an otaku. The media jumped on the case of man who had kidnapped, raped and murdered young girls, and kept them in his house, which was full of hentai manga, or erotic comics, the media then labeled this collection as “obvious signs” of an otaku.

Hikikomori are the latest addition to outcast groups in Japanese society. Some hikikomori are school dropouts whom refuse to subject themselves to Japan’s educational regime. These children fail to become a part of the in-group of the school system, in both academics and more importantly with their peers. Bullying and isolation are usually the major reasons for these people to leave school. A feeling of individuality can cause them to feel different, and therefore wrong (Note: The Japanese word Chigaimasu (Jap. 違います) means both to be different and to be wrong).

Hikikomori can feel that they have failed society, or that society has failed them. By withdrawing from society and denying its existence, they can concentrate on themselves by escaping to a fantasy society such as television, or the internet, a place where nobody can put one under the eyes of scrutiny.

There are an estimated one million hikikomori in Japan today. Hikikomori for whatever reasons feel isolated and therefore different, this leads them to remove themselvse from society, and withdraw into themselves. Dropping out of school is the major step in becoming a hikikomori. Instead of going to school, most of these children spend all of their time in their bedrooms playing video games, watching television, and participating in and other forms of self-satisfaction.

Hikikomori are different from otaku in the way they consume these forms of media. Otaku devour the media as a form of knowledge and as a way to make themselves part of a group. Hikikomori on the other hand use this media as a form of escapism. Television allows them to see the outside world without them being observed or tested. Video games are a way to interact without personal attachment, or true human interaction. Many hikikomori will use the Internet to communicate, a form of communication that involves no direct human contact whatsoever.

Japanese media and society has recently begun to concern itself with these children. It is caught in a paradox where it cares about these children, but at the same time praises devices like the Sony Playstation 2, which contains an Internet terminal, and a DVD player, both which can be used by a hikikomori to isolate themselves. These devices allow for entertainment, but at the same time the ability to conduct commercial transactions online, without stepping out of the house. It fixes people into their individual space.

The Causes

What causes these people to become they way they do? The Japanese education system is ideal for a state that means to create people who will work well in industrialized society, and contribute to the state by being a productive individual who works towards the common goal of prosperity for the nation.

Pupils are taught not to think but to accumulate immense piles of disparate facts that can be repeated on command but cannot be connected. This is not an accident or a lapse. Rote learning is the child’s next lesson is dependence. To think is an act of autonomy; to memorize the given is to rely upon authority.

This type of education system, rote learning, is measured by intense examinations that will judge if you have conformed, as well as if you have memorized the facts presented to you. The subject will either pass or fail; this does not judge individual achievement, or critical thinking, but instead to rewards the most conformist individual with entrance to industrial society.

Savage competition and force-fed information without the cultivation of critical thought go far to explain the character of Japanese graduates. The system’s demands-years in exam hell, every other student an adversary-produce not exploring intellects but the narrow, machinelike people we assume the Japanese to be by nature. Wholly focused on achieving the highest possible place in the hierarchy, they are unable to form healthy ties with equals-horizontal relationships. They are indifferent to most public issues because they are unnaturally inward-looking and (by official design) ignorant of large patches of their own history. Outside of a few conventional settings-karaoke bars are one-they display little sense of autonomy.

Otaku are the direct result of this type of education system. They are an extreme example of an authoritarian and militaristic rote learning system gone horribly wrong, an overextention of the education system. While these people consciously rebel against the society that has created them, they simultaneously conform to and exhibit the same tendencies and characteristics that the system has instilled in them. This paradox has created a people who are extreme examples of “The perfect Citizen”. One who unquestionably lives, works, and only thinks about the topic given to him/her. This phenomenon is concrete, as you would find it increasingly difficult to find an example of otaku outside the realm of the academic system.

There is no doubt that Japan’s educational system produces a dedicated workforce, and that these “corporate warriors” are the engine behind Japan’s tremendous industrial strength. Obedience to authority, instilled in people from the time they are small children, makes Japanese society work very smoothly, with far less of the social turmoil and violent crime that have plagued other countries. All this is on the plus side of the balance. But there is a minus side, which, like so many other modern Japanese problems, has to do with once-good ideas carried too far.

Hikikomori, on the other hand, also exhibit symptoms rooting from problems within the education system. These problems are from a different issue entirely. The Japanese education system is built upon the concept of the individual as a contributor to the society.

Standardized textbooks, uniforms, school rules, marching in lockstep around the school grounds, bowing in unison-these regimens were able to achieve what 350 years of isolation could not: a triumph over regionalism and individuality. It was probably Japan’s single most serious modern maladaptation.

When children start schooling in kindergarten, the teacher assigns the students into “kumi” or groups. These groups will stay together until graduation; they play together, eat lunch together and study together. The kumi system builds a strong basis for an “us” and a “them”.

A popular phrase develops from this group orientation called “nakama hazure” or to be cut off from the group. The children forge a special bond from their group, and don’t want to be cut off from the group, or they will feel rejected.

Hikikomori have often been rejected from their kumi, and have chosen either because of a feeling of failure, or a feeling of rejection, to withdraw. This kumi system is a major catalyst for the budding hikikomori, and most current information shows that these children had problems with being accepted into such groups.

School bullying, or ijime is a major issue contributing to the problem of hikikomori facing the Japanese education system. Ijime arises from the fact that accepting your role in a conformist society is one of the major issues growing up in Japan. Accepting your role as a member of a cohesive group is troubling for youngsters in the system. Due to the intense pressure to conform from kindergarten onward, many students resort to ijime. Any sort of difference is a target for ijime, from a difference in speech, or appearance, even excelling in a specific course is reason enough. Many times students who excel at a particular class will perform poorly so as not to stand out. There isn’t much a student can do against this style of bullying as the one who is bullied is considered the one to be at fault. Teachers indirectly encourage it, to emphasize obedience within the group.

Students who have studied abroad are obvious targets of ijime. So different is their upbringing that their classmates have coined a new word for them; kikokushijo or “Returnees”. Often times these students have to attend special schools to be indoctrinated back into Japanese society.

An interesting phenomenon in Japan is the way the class teaches itself, from after-school study groups, in class help, and self-assigned roles of class-representative, etc. This also involves some children trying to unconsciously involve the others into the group. If a child is expressing his or her own opinion, has her own aspirations, or doesn’t exactly fit into the group, she or he will be ostracized. With all of these separate issues combined, the issue of continuing to be an individual becomes increasingly difficult.

Hikikomori are the result of the unconscious rebellion against a system that creates robots. These children do not want to be part of a society that has no individual thought. Since they have no contact with school, they also loose sight of education, and complete social maturation. Here they fall in between the social requirements needed to live in Japan, but also lack the individuality to become an autonomous person.

Results

Now that the Japanese have a new out-group, the finger of blame can be directed as the source of an assortment of social problems. Japan is now looking at these “fringe-groups” as the source of the problems, instead of examining the make up of society and the educational system as the root of the issue. The media continues to pounce on every opportunity it gets to portray just how sinister and deplorable members assigned to this group are.

Until Japanese society begins to examine its own social psychological processes, it will continue to persist an environment that continually defines its own identity by reference to out-groups that it creates and reinforces.