what about the converse of this? what about producing a basic set of content and letting AI taxonomise or organise it and then create a design based on design models. It could work both ways. I guess ultimately the problem will be solved both ways and AI could create entire experiences based on specific domains. Also, for those claiming this will never work i would suggest looking at some of the amazing stuff being done with AI right now. This kind of problem, including optimising and making best usage of html/css is almost tailor made for AI.
Training an AI to convert design mockups into HTML and CSS
101–109 of 109 posts
Re: Training an AI to convert design mockups into HTML and CSS
#102Earlier quoted context omitted.
Does it matter if the code looks ugly or not? End users are not going to see the UI. If i have an AI in my toolbox that autoconverts designs to Code, within a minute. It would speed up my development time by a lot!
It matters because ugly code is probably also slow code (consider loading of unnecessary resources, and consider an excessively complex DOM). And probably also a page lacking accessibility.
> It matters because ugly code is probably also slow code
This is a very bad assumption, that's highly dependent on subjective and overly-broad terms like "ugly" and "slow".
Optimized assembly code is ugly, but it's also probably faster than what a human would choose to write.
Linguists used to have this human-understandably-sized set of speech units called phonemes. Once we abandoned those for finer-grained pieces that were too many and complicated for a human to grasp, speech recognition got better.
Understandable by a human may be a fundamental limitation on the power of our models.
Re: Training an AI to convert design mockups into HTML and CSS
#103The example mockup image shown is really unimpressive. As usual, the difficulty is in implementing the last 20% of edge cases and design subtleties, not well-defined black-and-white block layouts like the given example. It would've been simpler to build a classical program to handle that example, and the result would be much better and more predictably structured. It's like programming a self-driving car for a 2D gam…
Yes, this is typical AI weenie optimism. "We can handle many of the easiest cases, so in X years we will be able to do all the really hard things that we haven't even thought of yet!" No.
i dunno, we could equally say that your position is UI optimism. AI is new tech, certainly as it now becomes feasible to apply it to real problems. Given some of the challenges AI is solving right now this one is relatively trivial, HTML and CSS have specifications and there are well held paradigms and principals of User Interface design. The domain is very constrained so lends itself to AI very well. To say this is an exercise undertaken by the author and the very visible progress he has made is very impressive. If you read the entire article the work he does on recreating designs is even more so. Surely one of the goals of 'intelligence' artificial or otherwise, is to actually deal with "hard things that we haven't thought of yet" AI is solving (and unearthing) those problems right now. Imagine somewhere like squarespace where all you do is upload a design and website is produced as a result is that beyond the realms of possibility given this really impressive experiment produced by the author, definitely not. Id hate to think we had to contrive edge cases or fabricate complexity just so we can prove that AI cant sole every problem...
Re: Training an AI to convert design mockups into HTML and CSS
#104Writing HTML and CSS to implement a mockup is trivial for anyone with even a little bit of web dev experience. If you want high quality code, an AI isn't going to produce that. I can't imagine ever using something so complex, if I can implement the same thing in a few minutes. There's also further considerations when dealing with the real world. For example, you need to be aware of how to handle different accessibili…
If we think AI would not do awesome code, then we're ignoring the achievements of many of the AI bots. I saw an LOL bot outperform lots of world class videogamers. What makes you think a bot will not outperform any HTML/CSS frontend dev.
There is a huge difference between these two scenarios. LOL has a very clear objective function and feedback. Code quality is a metric that is hard to make precise, even humans have trouble with it.
Re: Training an AI to convert design mockups into HTML and CSS
#105Why are design mockups done in photoshop, when HTML and CSS can be faster and easier once you have the basic components??
you forgot to mention emacs..!
Re: Training an AI to convert design mockups into HTML and CSS
#106Earlier quoted context omitted.
If he just try to learn something it is unappropriate to start the article with this sentence : "Within three years, deep learning will change front-end development. It will increase prototyping speed and lower the barrier for building software."
In three years software like this would absolutely change front-end development. Already is front-end development constantly progressing every year. PSD to HTML services were outsourced a decade ago for a reason.
Hopefully an AI won't insist on halting the sprint to change build tooling for the 3rd time in as many years at least.
Re: Training an AI to convert design mockups into HTML and CSS
#107Earlier quoted context omitted.
It's not always that straightforward. I automated my job once, as a junior dev. My "job" (if you can call it that) was copying a web app's configuration off an Excel file, pasting it into a text file, converting it into XML by hand, feeding it into a program that converted it to SQL and finally running the SQL to create a database. Normally, I'd then have to tear it down again and start all over- because hand-crafted…
> you may think that this is a fair world where engineering talent is rewarded, but the truth is that most companies are not run by engineers and decisions are made by people who don't care Perhaps you should find a new place to work. There are engineering firms that value talent and capability. While it's still important to be aware of business needs and deadlines, that doesn't mean you can't have a solid engineerin…
>> Perhaps you should find a new place to work.
Oh, that's a few years now. I definitely did find a new place and the culture in that next place was fantastic :)
It's just that not all teams I worked with encouraged, or even recognised, time-saving automation. Some did, some didn't- it was a mixed bag. Maybe I'm wrong to say that "most" didn't. If I think about it, out of five teams in six years, two clearly didn't care, two definitely cared a lot, one I don't know.
I think the difference in culture had something to do with the way hierarchy worked in different companies. It just happened that in every one of those jobs I was the most junior member of the team (in the last one because despite having some experience, I worked with mainframe engineers with 30+ years on me :). But, some places appreciated the fact that I did my best to keep up and pull my weight, while others- it just didn't make any difference. I was the least experienced so I was expected to trudge through endless tedious until I had tenure (by which time I was expected to delegate all the tedious to contractors or juniors, I bet).
The effect was that in the teams that encouraged putting in the extra bit, everyone did. In the other teams, not so much.
Re: Training an AI to convert design mockups into HTML and CSS
#108This is what we are all working for. If we can automate the stupid stuff we do then we can work on other interesting stuff and automate it too. I don't understand why we aren't all working on automating everything that we do. Every line of code you pulled out of the code mine today is finite and given a business logic rule engine it's outcome could have been generated.
> automate the stupid stuff Stupid is subjective :) I'd be happy to automate backend development and focus on a top notch UX and great front-end experience.
OT but how do you feel about backend-as-a-service setups like Firebase, graph.cool, etc?
Re: Training an AI to convert design mockups into HTML and CSS
#109Earlier quoted context omitted.
Found the front end developer :D But all jokes aside, I have to agree with others replying to your comment that this looks to be a learning exercise. It shouldn't be berated or beaten with a club. Without this kind of curiosity, the tech industry wouldn't be where it's at today.
From the article: "Within three years, deep learning will change front-end development. It will increase prototyping speed and lower the barrier for building software." -- This is an article that somewhat breathlessly claims that AI will be the thing plugging the "last mile" problem in the coding of the front end. I think myself and other other folks chiming in here would be less skeptical if this "last mile" problem…
From what you posted:
"It will increase prototyping speed and lower the barrier for building software"
How does that claim that AI will be plugging the last mile problem? That extremely vague statement says only that it will increase prototyping speed and lower the barrier of entry.. neither of which hints at solving the last mile problem.