Monday, August 28, 2023

A modest proposal for decentralised CI/CD

 I am living 'oh that a man's reach should exceed his grasp' and I'd like to log my 'heaven', but just for Python web applications and documentation.

I've used cookiecutter-django to begin a practice project and to avoid the main problem I was going to solve, I've been thinking about how best to track the project, without using Jira or web services from elsewhere.  I acknowledge the lesson of 'xkcd's "there are thirteen competing standards, we should have one to unify them -> there are fourteen competing standards" but I decline it.  

I've been trying to get a CI/CD setup going and the amount of special tooling that I don't normally use seems crazy.  Here is an alternate concept that creates a lot of work but keeps all of it as far left as possible - the developers repositories.  This is at least partly inspired by this talk from Simon Willison.

Idea 1.   Test reports are output to a tracked folder and can be identified with the git commit snapshot they ran on.  They are always added to the repository (e.g. by pre-commit) whenever a threshold is met, e.g. at least when merging to the default branch.

The idea would be to smooth all merging to downstream repositories. If a repository has new commits, it is ahead, and if its tests are all passing its worth trialling a merge. If the merge candidate passes all tests, the new state can be committed; if the merge should be aborted. This will work in a lot of settings, but there are some risks* too.

Idea 2.   Issue tracking, changelogs, release notes bugs, wikis and other things are just documentation and can live in a subfolder of your documentation. Using the 'sphinx' documentation system, you already build the documentation for the project along with the project - the idea is to extend that with files for issues that are composed in the repo.

Some cool things you could probably implement with a little code in conf.py (which, given the state of the world, probably means 'someone somewhere is doing this and wondering why they can't get more github stars'):

- show from git log the commits, merges, open branches with links from summaries into more detail (e.g. -p to get patch summaries)

- show the current lists of issues, bugs, etc, using the toctree or other directives.

- show test results for each commit

If you want to be legends, you can further reorganize that into a single table integrate the branch names and the issues.

Idea 3.  Use playwright's ability to produce animated gifs of the latest look in the documentation; that will look killer.  Update these at pre-commit.


*risks: in a large project you will probably have some special resources that are not perfectly replicated in your development environment. For example, your production database will have a certain schema and you will need to update that schema in order to use the updated code. While its easy to to revert the code, and its possible to reverse the schema migrations, the migrations may have destroyed data that then also needs to be recovered. Other setup, like certificates or connections could be destroyed. Although leaving folders for local settings or migrations untracked seems like it would help, things like data migrations need to know their place in the migration history graph.  A git merge strategy setting will still allow additional migration files to be merged into the folders.  Unless you're postgres or sqlite, you don't have enough test coverage to be perfectly confident that what worked 'there' will work 'here.'

Friday, June 10, 2022

, existingly.

Heidegger's digestion of the existentialists squeezed them into some fairly accessible forms under inaccessible language. You might know the German 'Dasein' as the type of being that we are; beings that are alongside, that in maintaining themselves, engage with the world on their terms. I think (though no one serious will probably agree, so don't crib this for an essay) that all living things can satisfy a minimal description of Dasein.  A bacteria has thrown-ness, presses into possibilities, has ontological guilt (a finite genome that may be maladapted to the environment), being-toward-death in pursuing more life, more DNA, reproduction.  The DNA & protein technology of a bacterium is like the layer upon layer of practices, on top of layer upon layer of organic processes that define the kind of being that we are, which Heidegger calls Dasein. Dasein is  famously 'always-already in a world', and slightly less famously, 'We are our world, existingly.'  

Where Hegel talks about the self separating from world in a negation, Heidegger saw the affirmative property 'mineness.' Both are intended (I think) as referring to ongoing maintenance of a distinction, that engender further distinctions in the subject as the world is apprehended in an endless process of learning, of becoming familiar.  (It seems possible that Heidegger thought Hegel had dealt with this and so didn't spend much time talking about learning? He discusses breakdown cases, and finding the property of heaviness from the fact that the hammer is too heavy for a particular task, which is, I guess his version of the dialectical progression and arises from his emphasis on 'being' as the big invisible thing we're all doing in our way, rather than changing our being, becoming.)  

"We are our world, existingly" is emphasising that the "ways" we make our selves of are all apprehensions of the beings (things, tools, beings, other people) that are our world.  ('World' in Heidegger is really not far from the conventional usage - X's world is one of personal drama, or gardening.  World is not the absolute world, but a human's map of the world, a 'worldonculous', we might say.)

I think the modern "large models" approach that astonishes* us with apparent understanding of text meanings (Google) or generative models like DALL-E are (mini) attuned worldonculi; they are an articulation (jointing) of the background.  (I say 'mini' because they are trained (so far as I have seen) on data sets and the data sets are of similar type - video streams, audio streams, text from documents, or spanning two types - labelling text and imagery. This limitation is probably useful (for safety? certainly for learning how things work), but a concept 'tree' that is constructed from images or text, or spanning both, is very lightweight. As you climb a tree, you get measures of strength, stiffness, roughness, yield, and once those dimensions appear you consider them for other trees etc.)

But, suppose we have a very large model, trained with an enormous quantity of data of great dimensionality, that 'handles' things in the world including 'behaviours' - what would it lack with respect to being the kind of being that we are?  We have computational apprehension of a world, but something is needed that 'has' that world, 'existingly.'

I would put a lot into "existingly": the kinds of grounding practices that make life possible and that enable new life, more life over the very long term are part of it in what we know of ourselves.  Indigenous knowledge in very old continuous cultures is of that sort, and the 'radical freedom' of existentialism is not really known to it - human flourishing in a place is good, but has to be bound by the nature of the place.  There is a religious duty to not eat female turtles, for example, or not to take all the eggs from under a sitting mother bird. The duty is a duty of essence - it defines the type of being in the land that the land allows but only humans with that in their makeup at the level of religious commitment will e.g. die obeying it.  

Non-indigenous people may learn our living in much less grounded ways (and it is unstable - that our children's children might still be able to live exactly as we do is not something we aspire to), but the essence-binding religious natures we may adopt are more explicit and so more accessible. It is in the way dasein exists that it may take a stand with regard to itself - it has a world, and a knowledge of what within that world is 'mine' and it takes the risk of committing its existence to ungrounded knowledge.  Some themes recur - a dimension of moral life - that there is better and worse behaviour, better and worse personality, that judgment is needed but that good judgment is very merciful, that the self is fertile ground for mistakes, and so on.  None of these can be developed except as an engagement with others, through time.  But this is where ability to train on a world full of racism and sexism but exclude them from yourself begins.

So, the question of how to take a worldonculus and make it conscious remains very hard, and involves a lot of things that would not be more accessible in symbolic computation (that I know of) than they are in sub-symbolic.  I am inclined to think, with Robert Brandom, that the normative rules of what counts as knowledge and rationale can ultimately be mapped to some modal-logical description of the world as it is, up to a point. The harder cases are how normative rules about other people and clashing understandings of the world are resolved. Humans are hard at work negotiating this all day every day from birth and we (I won't speak for all humans, but even just me and my friends) already accept a very diverse slate of strategies, from parenting to bossing to voting to trading - there are many accepted templates, and (you will notice) acceptance of many of them is also actively being contested (both theoretically by thinkers and practically by doers).  

What do I think they should do?  Maybe explicit moral instruction would get somewhere - labeled speech roles; understand this is a bad person, this is hostility - now how would you do hostility to a bad person?  I don't know if a self-presentation is intrinsically necessary, or if communication and negotiation with an Other (or two or three) similar will spontaneously lead to it developing.  

(I just had to get this off my chest)

*There are plenty of critics of the achievements of these models as, for all the gargantuan computational engineering they require, are too 'lite' for use in the real world - the regular fatalities in 'self-driving' cars and the ongoing 

Monday, December 11, 2017

Thor:Ragnarok spoilers

One of the advantages of having absorbed a lot of very old media (the Bible) is the pleasure of recognising elements when they recur in popular culture.

Thor spoilers follow -

Hela returns when Odin passes on and reveals how she was his deadly right hand in his conquest of the nine realms, his first daughter and she revelled in the bloodletting as the Goddess of Death.

Compare this to the female personification of wisdom in Proverbs 8:22-ff. 'I, Wisdom, was with the Lord when he began his work, long before he made anything else. I was created in the very beginning, even before the world was set up.'...':30 'Then I was beside him, like a master workman, and I was daily his delight, rejoicing before him always, rejoicing in his inhabited world and delighting in the children of humankind.'

I doubt that the contrast with Wisdom was much in the mind of the creators of Thor:Ragnarok, but I think it makes a good illustration of the way that the Lord was not always or exclusively understood as essentially warlike, for all the wars his people were engaged in.

Monday, April 17, 2017

Top-mount Oliveri Monet 71U

I bought this unit second hand, with a set of mounting clamps as pictured. They won't grip the small flange available, and Oliveri claim that this is an under-mount sink. I don't want it undermounted, and don't think the design wants to be undermounted.  So, to use these clamps, here is what to do:. Cut down the body of the clamp and the bolt by about 5mm or 2-3 threads. I ended up with a range of sizes and most worked. If you start with 12, lose a couple to error, then you still have enough.  Bend the legs away from the biting teeth - they are going to press inside the curve of the flange and balance the torque produced by screwing down.  

Hard parts - getting a hole through the flange, driving the bolt through the hole without the thread stripping, getting the whole job done in a way that lets you seal with silicone and put the clamps on to hold it in place.  Getting the clamps down is sufficiently difficult that I chose not to undo them to silicone under the lip of the sink.  

Monday, September 26, 2016

Google Problematic

Press release, Google, 1 Apr 2017

As an important internet company, Google owns a lot of information, and let's be real, a lot of it is problematic: it is the sort of thing that knowing it will get on your conscience and keep you up at night: searches for hit men, questions about how prosecutions of accidental sex deaths failed, searches for grotesquely of every kind. And then there's everything else, everything we index, which is about everything. Also, we know who is searching for or uploading these things: people.  Parents, kids, serious people with responsibility for other people's lives, managers, leaders, judges, politiicians, the desperate, the mentally ill, the unstable. All with an expectation of privacy, despite the reality that everyone is tracking everyone around the internet: governments most of all, but also advertisers, credit raters and general internet intelligence gathering snoops. We at Google generally believe in transparency and being real.

So since 1 Oct 2016, Google's latest team, Google Problematic, has been emailing a machine curated digest each week of the online behaviour of concern to those it might concern: the people who will find it a problem:  your wife, your parents, your parole officer, your kids, your boss, your female subordinates or co-workers, your therapist.  We hope it has helped.  We certainly feel better taking what responsibility we can.

Saturday, September 10, 2016

The Fool Proof

If you are a gin enthusiast, you could do worse than laying out for Poor Tom's gin.  For Fathers Day I have been graced with a bottle of the 'Fool Proof' 52% Poor Tom's.  I have just had a dry G&T (equal parts making it 'dry') and holy kablooey in. A craniium.  Delicious, and nourishing.

Thursday, April 28, 2016

Love your work

You might not have picked this up, but my foolery extends to the workplace.  I have been through an escalating series of difficult conversations about my non-compliance, non-performance and general devil-may-care attitude to achievement.

This results from an accumulation of disappointment.  I thought I was going to be amazing the world with discoveries and inventions, but so far the only thing I've invented has been generally regarded as silly or incomplete.  Those who judge these things use criteria pretty far from those I would apply, and if I judge their performance I find leadership almost totally lacking except in the domain of domination, performance measurement - very rarely are we stopped from addressing clients' non-problems.  Someone once asked "could the leaders give us an overview of their understanding of the field we're in, just once a year?" and the answer was a round rejection of the very idea.  So my demotivation is not coming from nowhere.

Still, I was being pretty ridiculous.  So I went into a long weekend seething with resentment that I was being held accountable for my timesheets, a very weak tool measuring in painful accuracy, my attendance. My performance is also measured, but less frequently and with better-designed paperwork.

Then, I watched a lecture on Oedipus from Peter Sruck's Coursera.com course on Myths from Greece and Rome.  Oedipus' story is amazing. Peter brought out the question of Oedipus' responsibility.  Here was Oedipus meeting a man of his fathers age on the road and, over the issue of who should step aside, killing Lais (his true father) and his whole retinue. I had been thinking about the Oedipus complex of irresponsibility and wondering why I was so much closer to Oedipus than Oddyseus.  This moment crystallised it.  Even if I am in the right about the uninspiring leaders, the naff prescription of measures and mechanisms that are not applicable to my type of work (I am not manning a desk for a shift, I am solving problems), I still believe I should get out of the way of the silly old king when he asks.  And that gave me the key.

I have to love work charitably, not Libidinally.I am not going, ever, to get a glandular charge from doing what these guys ask.  It will always be disappointing. But I shouldn't make these guys die in a ditch defending their stupid practices and total lack of vision. There are times to represent that, but stepping aside, just going along a bit more, out of charity for their feeble age might be the answer I can live with.