I agree with much of the article (also as being a java developer for the last 17 years).
However, I don't think the problem ie necessarily Java Developers - I think they were just early adopters of the larger problem:
Frameworks.
Too much software these days relies on frameworks in order to 'get things done'. The knock on effect is that developers don't have to think as much about how things work (technical debt is at an all time high), and often code following 'best practices' handed down from academia/conferences/books that promote large abstractions which are often un-necessary.
It's similar to how C code would end up having pointers-to-pointers-to-pointers-to-pointers...
A whole generation (generalising/stereotyping, I know) is now coding, who have never grown up and learnt with basic logic gates and low-level code. The framework is the obvious choice because 'why would you want to write it yourself, if it exists' ?
(Ironically, many Javascript developers seem to be intent on re-inventing everything that's gone before, and it's somehow seen as innovation - go figure)
Grumpy old man ? Sure, I am. I'm also gladdened to see a recent upsurge in assembly language and other low-level coding, as this is what the world actually needs - less super-business-entrepreneurial startups, and more people able to actually solve real problems and not just hipster entertainment.
Interesting! The most important difference between frameworks and libraries seems to be the "don't call me, I'll call you" pattern. Frameworks call your code, while libraries get called by your code.
It's easy to see why "don't call me, I'll call you" leads to unreadable code. A piece of code is readable if you can understand how it works, not just what it does, and a big part of "how it works" is figuring out the control flow. Inverted control flow makes code less readable, regardless of any other benefits it might have.
Viewed in that light, it's hard to see how moving to better languages like Haskell would solve the problem. Functional programmers pass functions around all the time, so a lot of functional "libraries" are actually frameworks that insist on calling your code in complicated ways, making it conform to very precise types required by the framework. There's certainly no shortage of abstraction in Haskell code. As a result, it seems that most Haskell programmers don't even try to understand the control flow of their programs, because it's almost impossible.
I wonder if we could have a principled approach to programming that discourages the "don't call me, I'll call you" pattern, and encourages code that you can simply read from top to bottom. Do this thing, then do that thing, then do the other thing. It would probably be unlike anything that exists in academia today.
Strange, since I started using Django a while ago, I would say it encourages me to keep my own code a lot more organized. Are Java frameworks so much different?
As for functional programming, I only mix bits and pieces here and there. One thing I find quite discouraging about JavaScript, is that (in the code I have been reading) you can define a function within another function, and pass that on to other parts of the program. I have found it really difficult to work out what is going on in the code.
In a basic Python script, define the functions at the top then write something to call them at the bottom. Its easy to see where the code starts and ends - well a whole lot easier than looking for function definitions within other function definitions. It just seems unorganized to me. I know that it is supposed to be a benefit of functional programming that you can do so many things with functions, but I find it makes debugging difficult, especially at the start. Is Haskell better at keeping the code organized? I have never used it.
I really like your work, but not sure that it lies on the right side of the framework/library divide. For example, to realize the full benefits of your approach, you define a new programming language with "no escape hatches". That's the most extreme kind of framework thinking!
> Frameworks hurt sharing. I'd really like to give you this fork Jimmy, but you're gonna need a knife and plate to use it. The framework checks out all your girlfriends for you, the framework won't let anyone dirty get through, the framework will wait up until you get in, the framework will always find out were you've been, the framework keeps you healthy and clean. Frameworks embrace, extend and hold on to greedily.
We are not so much defining frameworks as we are new ways of computing. In that sense, it's not really something that will be ready tomorrow (or ever). I do use glitch as a library in C# for UI and compiler programming (hard to program very interactive UI and compilers otherwise), but that is with lots of escape hatches :)
Yes: for example the x86 MOV instruction is actually a family of instructions. You can see the leakage in the restrictions on which combinations of addressing modes you can use.
code that you can simply read from top to bottom. Do this thing, then do that thing, then do the other thing.
It's called "imperative/procedural programming", and has been around for as long as computers if not longer (in the form of instructions to humans). I think the problem in frameworks is not so much the inverted flow of control as it is the extraneous use thereof --- control flow that bounces around between many functions is difficult to follow in general. It all comes down to the fact that abstractions have a cost and benefit; the important point is when to make the decision to use one in which the benefits outweigh the costs..
+1 for the last sentence.
I get this nice "do this, then do that" understandability feeling when I work with python scripting of cloud services (e.g. AWS with boto). You get all the powerful apis and OOP mechanics boiling down to readable procedural code with helper functions.
There are several examples of simple libraries in Java that makes you realize it is possible to provide solutions to problems without the "culture" that predominates the ecosystem.
For example Simple (http://simple.sourceforge.net) is a great XML serialization framework that does very well what it is supposed to do.
And there are others I haven't used, but you look at the API and think, this is what I want.
I was very impressed with jDBI (http://jdbi.org/) even if we could not use it because of Maven.
Just had a look at jDBI and it doesn't exactly look like a simple library to me. There's a lot of run-time annotation processing.
Have a look at jOOQ if you haven't seen it yet. There is a pre-compilation stage which generates DAO for you and they can then be used with static type guarantees.
That pre-compilation step, though... How does that play with the IDE? I'm looking into various (more) modern ORM/DB-talky-to-things, and checked out jOOQ but was turned off by that pre-compilation step. I like having my errors bright and red when I type.
I also don't like code generation. I'm not closed-minded to it, just don't like it if I can avoid it.
> but was turned off by that pre-compilation step. I like having my errors bright and red when I type.
Too bad. Once you have code generation built in your development process (e.g. combining things with Flyway: http://blog.jooq.org/2014/06/25/flyway-and-jooq-for-unbeatab...), I'm sure you'll get a hang of how things work with jOOQ, and most importantly, you'll get your errors bright and red when you type (or when you build)
> That pre-compilation step, though... How does that play with the IDE?
It doesn't integrate with the IDE and I am not too fond of it either.
But if you want ORM as a library in Java, I don't think there is any other option possible. Java has no meta-programming support, and annotations are going to be processed at run-time which is even worse than pre-compilation.
I don't see any reason it shouldn't be possible to support pre-compilation steps as part of the IDE. Unless you are actively editing the bits that are compiled, it seems like this should work, but I am not an expert on what goes on behind the curtains of IDEs.
The main advantage is when doing something like this is Haskell the code you pass around can't have arbitrary side effects, so you don't actually need to worry so much about where it is called.
Yeah, I agree that having no side effects helps quite a bit. But you still need to understand control flow to reason about time and space complexity, which are a sort of "unavoidable effects".
If "it is almost impossible to understand control flow in Haskell" as you posit, then how are so many people making time and space efficient Haskell libraries?
> As a result, it seems that most Haskell programmers don't even try to understand the control flow of their programs, because it's almost impossible.
The abstractions used in Haskell code almost always use existing and already well understood abstractions such as Monads, Monoids, Functors, and Applicatives. I most certainly have to understand my control flow in Haskell programs and it is not impossible.
> There's certainly no shortage of abstraction in Haskell code.
I've typically found said abstractions to be sound and needed (or at least justified) however.
Unless you're writing something very simple, you're going to end up re-inventing portions of a framework in order to do what you need to do anyway. In which case you're going to be wasting time. If you're trying to write a complex web application it isn't a good use of your time to implement templating/declarative views/JSON serializers/etc in a low-level language. It's not laziness - it's efficiency.
Low level systems programming is good for low level systems code - code where you have to interact with hardware directly or quasi-directly. If you're doing anything higher-level, like, as I saw here recently, a web framework in assembler, it is certainly a good intellectual exercise. But it stops there. You're going to again and again find that you want a lot of the functionality that a modern, higher-level web framework provides without having to implement it yourself.
Good programmers exist at every level of abstraction. The question isn't whether or not you're using frameworks and a high-level language but whether you're maximizing your language and your toolset and creating the most elegant, bug-free code you can with your time.
I do agree with the author's sentiments about OO, but that's just because I started to use Scala and I've started to really become allergic to state and mutability, recognizing that your code is so much easier to understand and write when you put all your state in one place - like the RDBMS or the filesystem - and your code doesn't need to worry about its own state.
OO works fine for plenty of people as long as you can design your object graph in a way that's not too convoluted, littered with useless or empty classes, and doesn't have too much concurrency.
You're going to again and again find that you want a lot of the functionality that a modern, higher-level web framework provides without having to implement it yourself.
That claim sums up much of what I dislike about frameworks.
In the marketing brochure, it’s your perfect dream home with everything you could ever need in a single, neatly integrated package. You get someone to do a survey to check for major structural flaws, you spend a couple of hours checking that all the essential utilities are connected up, and before you know it, you’ve made the biggest purchase of your project’s life. A week later, you’re sitting out on the deck, enjoying a cocktail with your glamorous friends on a warm summer evening as your baby sleeps soundly in the room upstairs, just like the models in the photo.
A fortnight after they hand over the keys and you move in, you often realise that what you really bought was a place that had 80% of what you needed. Unfortunately, to reach 100% you would have to knock down a few walls and rewire the whole building, costing at least half as much again as you already paid, so instead you start running extension cables from sockets in one room to appliances in the next, resort to flaky WiFi networking, and install a sofabed in the lounge because you couldn’t quite fit a double in what was supposed to be the spare room.
Over the next few months, you realise that it’s also really inefficient to run the place, because the guys who built it went for all the attention-grabbing features that would push up the price they could get, but what you really needed was energy-efficient appliances and a good HVAC system based on renewables. Also, the soundproofing between downstairs and upstairs sucks, and that baby who was supposed to be asleep during your summer party wakes up every time someone opens a new bottle.
Three years later, when you’re surprised to find you also had twins, there is nowhere to put the extra bedroom you need. A little after that, you have to move to a completely new place, because there’s nowhere to install a lift for your elderly parents, and even if there were you don’t have the three-phase power supply to run it.
Except you can’t, because in the few years since you moved in the area has become run down, the value of your place has sunk like a rock, and there is so little interest in buying it for anything but the land it’s on that you’re in negative equity and can’t afford to move.
Or, you know, use languages that take OO to its logical conclusion, like Ruby, Smalltalk, or the right answer in a concurrent environment, Erlang.
People hate on OO either because they're being edgy or because they are frustrated with bad implementations of its ideas.
OO is the correct way of thinking about things in the real world, because that is how disjoint systems communicate. Internal to an actor, use all the functional programming you want (and you should, because it's really good for that purpose!).
I disagree OOP is the correct way of thinking about the real world, and I think the software industry is starting to realize it was an overblown fad too (and moving on to the next fad, of course), BUT...
... I agree Java is not a particularly good example of OOP. It amazes me that so many people complain about Java "forcing OOP on everything". If I read framework source code, I see very little OOP in there. It's mostly procedural-with-classes. Maybe we've redefined OOP to mean "whatever Java is doing". Which is nightmarish, the worst of all worlds.
> It amazes me that so many people complain about Java "forcing OOP on everything". If I read framework source code, I see very little OOP in there. It's mostly procedural-with-classes. Maybe we've redefined OOP to mean "whatever Java is doing".
We kind of have, though it started pre-Java. "Procedural-with-classes" pretty much became "OO" from C++, and Java got it from there.
OO is a good way of approaching many real-world problem domains, but most OO programming languages implement OO in a way which has a significant impedance mismatch with the way in which OO is a good way of approaching real world problems.
I'm increasingly unconvinced that OOP (in terms of language features, even in the Ruby/Smalltalk sense, much less the Java/C++ sense) is actually an important tool in the mapping of OO thinking about systems to actual concrete programming, and that other approaches, like REST-based (not particularly HTTP-based, but the architectural style) SOA where the individual components are functional might be better -- traditional OOP, IMO, promotes tighter coupling than you real want in an OO system, and then requires elaborate patterns to mitigate that into slightly looser coupling than the language structures naturally promote.
Well, part of the problem is that we've all accepted that OOP is the best way of modelling the real world, but there is no convincing proof. Maybe experience? I don't know. We do know modularity is valuable. We know OOP is a way of achieving modularity, but is it the best way? And is OOP-like-in-Java the best way of doing OOP?
OOP like many Java (and C++) programmers do it is certainly not what Alan Kay was thinking about (in his own words!). But that's merely an appeal to authority. Maybe Alan Kay was mistaken and the Java folks are right. So the other part of my problem is that OOP is not a well-defined concept at all. I cannot begin to agree OOP is the best way of thinking about the real world until we can determine what OOP is; a definition both a Smalltalk dev and a Java dev can agree on.
Ok - so in a web application I need routing/binding of controller functions to URLs/HTTP methods, the ability to interact with an RDBMS and parse a result-set into an intermediate abstraction, the ability to take in the results of a POST or PUT and parse it/validate it/sanitize it, the ability to emit JSON....and yet all of this functionality is baked into my web framework. I can either use a library for each one of those functions or I can just use a light-weight web framework and get all of them automatically. Most likely the framework is just a collection of libraries anyway - Scala Play Framework uses, for example, Anorm for database querying, Slick for higher-level ORM, Jerkson for JSON serializing/deserializing, Twirl for templating, etc, etc.
Having imported those libraries, in the end what I write is going to look a lot like what a framework asks for if it's even remotely well-defined.
I think the advantage of using each of those libraries instead of a single unifying framework is that you put them there and understand why they are there, instead of relying on defaults. Also, there is less risk of framework-induced coupling -- sure, many frameworks are configurable, but try to move away from their recommended components and you are asking for trouble...
Also, there is less risk of framework-induced coupling -- sure, many frameworks are configurable, but try to move away from their recommended components and you are asking for trouble...
Exactly. If you built your own system, choosing a good library for each major recurring task, then you inherently have both an understanding of how the overall system fits together and a degree of isolation between the libraries. If your requirements evolve beyond the scope of a library you’re using, or if a library loses popularity and is no longer maintained, you can probably swap in a replacement to serve an equivalent (or updated/expanded) role relatively easily.
Of course, the price you pay is that you do have to write some glue code to connect up the libraries where perhaps a framework would have provided that for you. This has potentially interesting implications for both the way good library APIs are designed and the scales of libraries that are or aren’t worth the overhead.
This also has huge implications for language design as a whole, and personally I think this is one of the reasons why Java is infamous for its verbosity and boilerplate. I also think it’s one of the reasons why common idioms from functional programming have increasingly been pushing into mainstream languages that aren’t necessarily functional programming specialists: functional idioms offer new forms of glue, and in particular, they are often well suited to writing “slightly customisable” algorithms and adapting more substantial pieces of code so they can be used together.
But good frameworks are essentially collections of commonly needed libraries (to solve whatever problem the framework is concerned with) wrapped in (hopefully thin) unifying API.
Assembly, and Low Level Coding is the exact opposite of solving 'real' problems. Your going low level to create problems that didn't exist, as opposed to working on the core issue(often business related) your software is trying to solve.
Languages should allow you to focus on creating software that fixes a real world problem, as fast and cleanly as you can. That doesn't include bit flipping, or bit operators or anything like that unless required.
People should be working in super high-level languages like Haskell, OCaml, F#, Clojure, or languages that fit their domain perfectly. The goal is to fix problem the as concisely and quickly as possible.
There are many goals and many problems. "People" should use the right tool for the job. I don't want to try to write performance critical code or low-level code (OS level, drivers, critical libraries) in a language that seems almost designed to resist my efforts. Similarly, most people don't want to write business critical processing in C.
Writing low-level code is important for writing the underlying libraries for those high-level languages, e.g. the run-time systems or the big integer routines.
While it is important to know low level details if you are going to understand how the high level language is built, it is not necessarily important to know how the high level language works if such details don't readily aid you in your task.
It might be nice and it might be cool, but it doesn't matter so much. Or to put it another way, how many people learned how tracing JITs work to do JS form validation? Would it matter if I did/didn't know this? Would it make a better form validator? Eh.
There seems to be some limit to the benefit that knowing such LL details provides. It's icing, but it is most definitely not the cake, and when your time/budget/skills are limited, you want cake, not icing.
Who gives a shit? Computers are cheap, programmers are expensive. You're probably deploying onto Amazon anyways, so don't kid yourself.
In the vast majority of cases, it doesn't help your average coder one bit to know about whether or not a particular type of shift on their platform is arithmetic or logical, or how their cache is structured (again, running in a VM, lolzors).
There are cases where it matters--and surprise surprise, most of us don't work in those fields.
I more or less agree with your post, but take issue with a few things:
- Maybe I come from a different kind of academia, but in my CS education they didn't force "large unnecessary abstractions". Mind you, they didn't teach Java, C++ or any particular language. Learning programming languages was something you did in special optional classes or on your own.
- It is fascinating and instructive to learn low-level languages such as assembly, which I enjoyed learning in my spare time, but I don't think that's "what the actually world needs". The world needs reliable, non-bugged software that does what we want and is easy to understand, maintain and extend. Certainly not something that would be helped by coding in assembly...
To clarify: I believe it's immensely instructive to learn about low-level stuff. It should probably be mandatory in a good CS education. But it's not a good idea, in general, to actually build programs by manipulating low-level language. Sometimes it can't be helped, of course, by I assume we're mainly discussing application software here.
> A whole generation (generalising/stereotyping, I know) is now coding, who have never grown up and learnt with basic logic gates and low-level code.
This is something that the education system could fix. I believe that we should be teaching kids, starting in elementary school, things like binary and boolean logic, simple circuits with gates, etc. and then move up to memory, instructions, basic CPUs. Everyone should be exposed to some Asm programming. Higher-level langauges, OOP, etc. can be left to those who choose to go into computing as a career, but I think that due to the massive reliance that society has on computers and the influence they have on our lives, everyone should at least have a fundamental understanding of how these machines work.
This should also fix the problem with developers creating monstrosities of abstractions, since they'll have a better understanding of when it's necessary --- this is similar to the difference I've observed between C and Java code, where the former tends to be written far more straightforwardly and with fewer extraneous abstractions than the latter.
> more people able to actually solve real problems
Agreed 100%. It seems far too much time (human and machine) has been spent on creating artificial problems via some sort of abstraction and then trying to solve them (often creating more abstraction in the process), instead of focusing on the fact that computers were invented to solve real problems.
But as things like http://en.wikipedia.org/wiki/Java_4K_Game_Programming_Contes... and the various Java demoscene releases show, the langauge itself is likely not a major source of the bloat, but the culture - it's possible to write simple, efficient Java programs too.
> This is something that the education system could fix. I believe that we should be teaching kids, starting in elementary school, things like binary and boolean logic, simple circuits with gates, etc. and then move up to memory, instructions, basic CPUs. Everyone should be exposed to some Asm programming. Higher-level langauges, OOP, etc. can be left to those who choose to go into computing as a career, but I think that due to the massive reliance that society has on computers and the influence they have on our lives, everyone should at least have a fundamental understanding of how these machines work.
I think this is backwards -- the high level understanding needs to be more universal, and the low-level left for those who seek to make a career out of it (though I think the high-level part that needs to be universal is even higher level than actual programming and is more basic system analysis, though combining it with some programming, including basic DB implementation, would be a good way to make it concrete.)
However, I don't think the problem ie necessarily Java Developers - I think they were just early adopters of the larger problem:
Frameworks.
Too much software these days relies on frameworks in order to 'get things done'. The knock on effect is that developers don't have to think as much about how things work (technical debt is at an all time high), and often code following 'best practices' handed down from academia/conferences/books that promote large abstractions which are often un-necessary.
It's similar to how C code would end up having pointers-to-pointers-to-pointers-to-pointers...
A whole generation (generalising/stereotyping, I know) is now coding, who have never grown up and learnt with basic logic gates and low-level code. The framework is the obvious choice because 'why would you want to write it yourself, if it exists' ?
(Ironically, many Javascript developers seem to be intent on re-inventing everything that's gone before, and it's somehow seen as innovation - go figure)
Grumpy old man ? Sure, I am. I'm also gladdened to see a recent upsurge in assembly language and other low-level coding, as this is what the world actually needs - less super-business-entrepreneurial startups, and more people able to actually solve real problems and not just hipster entertainment.