Building Site, Charing Cross Road

Most of this is hidden behind street-level boarding. You have to be in the Gents WC in Foyles to get this view. I don't know why Foyles has a loo, but I've needed it a couple of times and I'm darn grateful it has one. The urge to go interferes terribly with the concentration needed for serious browsing!

How To Run a Python program? And Why Is It A Good Thing?

Isn’t that a question with an obvious answer? Not so much. Answering it takes us to the heart of how Python works.

Write a Windows program in a language such as C#, and the IDE compiles to a self-contained .exe file (as much as any .exe is ‘self-contained’ when it makes calls to all those Windows DLL’s). My reflexes have been developed on languages like that: compile and run.

So my first thought, having got code that worked as it was supposed to, was to make an .exe file. It was then I found out that PyCharm doesn’t compile Python scripts as, say, a C# IDE does. If you really want an .exe file, you have to find a third-party application to compile the scripts and wrap in all the dependencies. The most popular way of doing that is to use Py2Exe. That only works on Python 2.7, and hasn’t been upgraded to handle the 3.x versions of Python. Lesson in there somewhere.

The lesson is this. There are two Pythons. One is the scripting language. The other is a program that runs those scripts. That’s why the command line prompt is ‘python mystuff.py’. It’s telling the Python program to run the Python script in ‘mystuff.py’.

The Python program on Windows converts a .py file into something Windows understands. The Python program on OS X converts the same .py file into something OS X understands. So .py files are portable. mystuff.py will run on any computer with a Python program. (Given Python version compatibility.)

Browsers work in the same way. You download a web page full of all sorts of HTML and other programming junk, and the browser interprets it all in terms the operating system understands and throws the result at the computer to run.

Which means that ‘everyone’ has to have a Python install to use a .py script. Well, okay. Would you expect to be able to ‘run’ an Excel workbook without Excel? You need Excel to be installed.

So how do I run the program? Set up a .bat file with the command line in it, and put that in the Start Menu. Click on it, and it will start up like a ‘real’ .exe program. Which is all I need.

Annual St-James'-Park-Lake-Covered-With-Green-Stuff Summer Photo

Yep. It's that time of year again.

Learning Python by Writing An Application (2)

I haven’t written high-security web applications that link to secure back-end databases, so I’m not talking about those. I’m talking about regular, runs-on-your-computer applications with a UI.

Real programmers know that their command-line utility never hangs and has a try...else to make it stop at the first sign of trouble, so they don’t need it to talk to them. I hate it when a program sits there seemingly doing nothing, I want some feedback. There’s a trade-off between speed and user dialogue. No, not progress bars. Progress bars require sizing the job first, so that the progress can be shown as a percentage of the total. Ever noticed that if the number of files you’re copying is large enough, Windows suddenly takes an age doing some mysterious preparation? That’s what I want to avoid. The only way round that is a running commentary. I don’t want much, maybe something like

Scanned N directories, copied M files to (destination)

updated every 10 files. When it’s syncing, I want to see

Scanned N directories, deleted M files from (destination)

(Real Java Programmers would write a log file of every action, along with the time it happened. I’m so glad I’m not a Real Java Programmer.)

Tkinter is notorious for updating at random. I don’t think this is Python’s fault, but the way the OS and hardware parcel out the tasks. So by the time Tkinter gets a moment to display the first confirmation, the whole task might have been done. To get round this, I called my first thread! I’m a VBA-basher in my day job so threading is something exotic and mysterious, like (insert inappropriate analogy here). It solved the Tkinter delay problem nicely.

My program has a button to choose the source directory, another to choose the target, one to Start and one to say Stop. Think about the UI logic. The Stop button should not be active unless there’s copying or syncing going on. The Run button should not be active unless there are valid source and destination directories.

What happens if I copy a file and there isn’t enough spare space on the target disc? At the very least I’m going to get an error message, and so I’d need to handle that. Or I could test for spare space and stop copying when I reach some kind of limit. That’s what I did: read in the spare space at the start, store that in a variable, subtract the copied file size from that value, and stop if the file to be copied is within some limit of the calculated space. Using the constant cuts down on filesystem calls.

What happens when there’s already a file with the name of the being copied BUT the size is different? The program needs to add a suffix to the filename, but then it has to test that that filename doesn’t exist, which sounds like a loop. And don’t forget to put a numerical limit on the number of attempts just in case someone used a silly file-naming convention.

A lot of programming is testing for and resolving edge-cases. For instance, the destination directory should not be the source directory, nor a subdirectory of the source directory. I got the first condition on the first pass, but only thought of the second when writing this. (Documentation reveals many improvements.)

This is all exception and edge-case handling.

In my experience, the actual business functionality, the code that does what you want done, is about ten per cent of the total code. Variable declarations and syntactic seasoning is about another ten per cent. Setting up, controlling and making the internal logic of the UI consistent, will be about twenty-five percent, depending on how complex it is.

All the rest is handling edge cases, data weirdness, potential errors, null values, and other stuff that might trip up the program.

I’m not kidding.

If half your code isn’t testing that things that need to be there for the next step to work are actually there, or for missing values, forcing stuff to be strings when the function expects strings, putting limits on loops in case something you have never thought could happen does happen and the terminating condition is never reached... if you’re not doing all that, and you’re not writing in sensible defaults for when the user has to make a choice, and you’re not testing for when the user does something really dumb because they’re distracted, or walks away for five minutes, or all those other things you don’t want users to do because then your life gets difficult... then you’re going to produce a program that people are not going to use twice.

The difference between a professional and an amateur is that the professional writes all the edge-case and exception handling they can think of. An amateur thinks ‘Well, the user will just have to...’.

Don’t forget to put in a default drive for the source and destination directories, or the getdirectory() function will dump the user deep down a file tree it will take a boring time to get out of and get to where the user should have been in the first place.

Programming isn’t about writing code to do stuff. That’s the easy part. It’s about writing code that makes sure the user and the computer don’t get in the way of the stuff getting done.

Learning Python by Writing An Application (1)

I don’t read Slashdot as much as I used to, and so I have no idea if people still ask “What’s the best way to learn a new (programming) language”? The answer was and ever more shall be: read a decent introduction to it, then write an application with it. Not toy examples, but something you will use. Why? Because then you will have to get to grips with the file-handling and string-processing functions, as well as some basic UI functionality, as well as the specialist libraries you need for the task.

That’s what I’m doing for Python.

Assume you have a bunch of files in a high-level directory called MyFiles, on the drive of your working computer. You want to make a backup of this from time to time. If it’s small, the simplest thing to do is copy MyFiles onto an external drive. With a lot of files that could get time-consuming, so maybe a backup program will do. Except that backup programs often create their own special file structures, databases, or at the least, zip everything up. I don’t want to use a special program to manage the backup: I want to use Windows Explorer.

What I need is a clever copying program. I tell it the source directory and the backup directory, and it recursively scans down the source tree, copying only the newfiles and the new directories it finds. Then, if I’ve asked it to, it removes any files from the backup directory that I deleted in the source directory since the last backup.

That’s the sort of thing that Python is good for. So I downloaded JetBrains’ PyCharm Community edition (because free) and started. I know how to program, and I understand basic Python syntax, but what I don’t have is a lot of familiarity with all the libraries, and especially Tkinter, which is what I needed for the user interface.

How does one cut code without knowing all the libraries? Slowly, with various reference websites open in the browser. That’s what manuals are for.

This illustrates the difference between being a programmer, and being a productive (insert name of language here) programmer. Because I’m a programmer, I can break a task into activities the computer or user needs to do (‘get the spare space on the destination drive’), and it usually turns out that the tasks I identify correspond pretty closely with the functions available in the system libraries. Programmers tend to think alike.

I understand control flow, arrays, variable types and declaration, threading, and use of functions to make code readable and maintainable. (I know, you want to hire me already.) I learned that writing FORTRAN, C, and VBA. Writing long chains of macros and the queries for them in Access is programming of a kind as well, as is constructing chains of raw SQL. It’s all about organising the resources of the language to get done what you need to get done.

Faced with a new language, I look for how those things are done, as well as any other cool tricks or idiosyncrasies it has. And, because I’m a programmer, I Read A Freaking Manual first. (This approach will not work with LISP and other such languages. Those really do need to be learned from zero, or you will miss the point.) On a meta-level, I’m Dreyfus-Proficient. That’s the engineering training. I’m just not slick.

All that said, I wouldn’t go for a Python job because one of the requirements is muscle-memory of the main system libraries, and familiarity with whatever the job-specific libraries are. I don’t want to be one of those guys who bluffed about their skills.

I have the sneaking suspicion that I’ve written VBA in Python, and that a Truly Pythonic Programmer would do the whole thing in two lines of code, one of which is a recursively self-referential array of filenames, and the other of which would implement a beautiful wxPython interface in less than twenty characters.

‘Pythonic Programming’ is a Thing. It’s a style. TPPs can do in a line what RJPs need six hundred lines to do. But becoming a TPP takes serious study, not to mention the right examples.

Next post, I’ll talk about the program.

April / May Diary

April and May are not my most stable months, because birthday. I don’t know why, because I’m not regretting the past, nor do I wish to shut the door on it. My only problem is an extra inch on the waistline and the difficulty shifting it.

April started with Easter. So there was that. The diary says I had a cold at the start of the month, because it was effing cold.

I was at the time doing steep incline walks on the treadmill and focussing on using the hamstrings to move my legs (it’s not as easy as it sounds). Then I suddenly got a painful lower back, blamed something else I might have done, and a couple of days later realised, yep, it was most likely the hamstrings. Another visit to Petra the Sports Masseur, who inflicted six kinds of pain with her elbow and pronounced me cured. Which I was. Not doing steep incline walks again.

Towards the end of the month, what with Sis and I alternating weeks to get colds, we hadn’t had our monthly supper. So I jumped on the Interwebs... if you want to eat at Gymkhana starting between 6 and 7 in the evening, you have to book about ten weeks in advance. So I did that, and booked us in at Merchant’s Tavern for the last Friday of the month, and Pizzaro in Bermondsey towards the end of May. Pizzaro do a fried dolche de leche which may be one of the best desserts I’ve ever tasted.

We got an exciting works day trip down to Cardiff towards the end of April, and very pleasant it was too, on one of the new trains and with very few people on board. I spent most of the trip going over and over the same three or so pages of Nik Weaver’s Forcing for Mathematicians, and finally came away with a much better understanding of how Cohen Forcing works. It’s much simpler than all those complicated derivations of Boolean-valued sets would suggest. That’s the subject of another essay entirely. I vowed I would not pass off this mortal coil until I understood Cohen Forcing, and now I do.

‘They’ re-located us again in May, to an ‘Agile workspace’ along London Wall. Suffice to say, I have to find an armchair (!) when I need to get my coding flow on, and spend much of the afternoon at a sort of dining counter (!!). The actual desks and chairs are unusable. As are the second screens. So I’m back to using the laptop screen, no mouse and no external keyboard. Do you know how insensitive the trackpads on a cheap HP corporate computer are? There’s a clue in the question.

For my sixty-fourth birthday, I visited my friend in Utrecht, as I always do. On the way over, I made a side trip to Zandvoort and had a nice steak at Vooges, one of the many restaurants along the beach. Saturday the weather was grey, we walked around Amsterdam, had lunch in the Cafe de Prins, and supper in the Restaurant Griftpark in Utrecht. Sunday the sky was blue and the sun was hot, so we walked around the canal and had a light lunch at the Louis Hartlooper complex, and a few hours later I made my way back to Schipol and a slightly delayed KLM flight.

I saw Deadpool 2 at Cineworld; S4 of House, and the Jack Reacher movies on my DVD.

I read Tom Holt’s The Management Style of Supreme Beings; Robin Fleming’s Britain After Rome; Slavoj Zizek”s The Courage of Hopelessness; Will Storr’s Selfie; Filip Springer’s History of a Disappearance; L David Marquet’s Turn the Ship Around; Nik Weaver’s Forcing for Mathematicians; Michele Friend’s Pluralism in Mathematics: A New Position in Philosophy of Mathematics; Øystein Linnebo’s Philosophy of Mathematics, and his co-edited collection New Waves in Philosophy of Mathematics; looked at the book of Stik graffiti I bought a long time ago. The commendation goes to History of a Disappearance: it's not a person who disappears, but a Polish mining town. It's actually quite moving at the end.

I finished and tested the MVP of my smart sync Python program, about which a couple of posts later.

I’ve been looking at G-Shocks for a while, but never quite found the right one. Side note: for a Real Watchy G-Shocks are in the Premiership of watches, not at the top with Rolex and AP / PP, but Premiership material. Then Amazon did a deal on the AWG-M100-1AER, which I decreed to be my birthday present to myself and snapped it up. Amazon delivered it the next day. Non-Prime. Not bad.

And May ended hot. So that was okay.

Like minded people: WTF?

Read any list of advice about how to improve your life and ‘finding like-minded people’ will appear. It’s a guaranteed sign of a half-assed list produced by someone with little knowledge of the human condition.

My birth year cohort was about 450,000 men.

I’m a lifetime bachelor. According to an ONS dataset I can’t find again, that’s one in ten of my birth year cohort. 1 of 45,000

I have an undergraduate degree from the mid-1970’s. Counting in the polytechnics, one in five of my cohort got one of those. 1 of 9,000.

I have a postgraduate degree. One in ten undergraduates did that in the mid-1970’s. 1 of 900.

I’m still working. Half my age cohort are doing that. 1 of 450.

I’m long-term sober. I can’t remember the last census count from AA, but it was less than 100,000. (Really. And even then, that’s a gross over-estimate of the number of ex-drunks with long-term sobriety). Out of an 18+ population of around 50 million. Which is 1 in 500.

So that’s me. Unique in my birth year.

We’re all equal, right? Well, if you want, but we’re all different. Some of those differences matter, and many don’t. Being a lifetime bachelor matters (as does being MGM’d as a baby, but that’s another story). That my degrees were in Philosophy of one kind or another, mixed with chunks of mathematics and logic, matters. Philosophers are not as other people. As for being long-term sober having worked the programme and done the Steps, you can’t even imagine the difference that opens up.

This is not a play for tragic status or for sympathy. I don’t want either. There are a bunch of other people whose experiences, on the way to wealth, artistic recognition or athletic success, separate them from everyone else except the other fifty people who went through the same wringers. I have nothing in common with them either.

My question is: who the frack are my ‘like-minded people’?

I have some friends. We can communicate. I think they might be insulted by the suggestion that they and I were ‘like-minded’.

So I have another question: why are ‘like-minded people’ important? Or is it a code? For, you know, being gay. Or fascist. Or an Aston Villa supporter. Or a Conservative in Rotherham. Or a train-spotter. Something that give my identity a distinctive flavour.

My identity, such as the poor worn-out thing is, has no particular flavour. So maybe there are no ‘like-minded people’ for me.