Edge and corner cases: AI’s conceptual simplifications and potential complications

Whether or not (and if so, how) to account for rarely encountered implementation variables and combinations in hardware and/or software development projects is a key (albeit often minimized, if not completely overlooked) “bread and butter” aspect of the engineering skill set. Wikipedia seemingly agrees; here’s a relevant excerpt from the entry for edge cases:

An edge case can be expected or unexpected. In engineering, the process of planning for and gracefully addressing edge cases can be a significant task, and yet this task may be overlooked or underestimated. Non-trivial edge cases can result in the failure of an object that is being engineered. They may not have been foreseen during the design phase, and they may not have been thought possible during normal use of the object. For this reason, attempts to formalize good engineering standards often include information about edge cases.

And here’s a particularly resonant bit from the entry for corner cases:

Corner cases form part of an engineer’s lexicon—especially an engineer involved in testing or debugging a complex system. Corner cases are often harder and more expensive to reproduce, test, and optimize because they require maximal configurations in multiple dimensions. They are frequently less-tested, given the belief that few product users will, in practice, exercise the product at multiple simultaneous maximum settings. Expert users of systems therefore routinely find corner case anomalies, and in many of these, errors.

I’ve always found case studies about such anomalies and errors fascinating, no matter that I’ve also found them maddening when I’m personally immersed in them! And apparently, one of my favorite comic artists concurs with my interest:

That said, I’ll start off with a bit-embarrassing-in-retrospect confession. Until now, as I was researching this piece, I’d historically used the terms boundary case, corner case and edge case interchangeably. Although Google search results reassure me that I’m not unique in this imprecision, the fact that there are multiple distinct Wikipedia entries for the various terms (along with the closely related flight envelope) has probably already tipped you off (those of you not already more enlightened than me, to be precise) that they aren’t the same thing.

Here’s a concise explanation of the difference between an edge case and a corner case:

Corner cases and edge cases are different things even though they are commonly referred to as the same thing. Let’s formally define both:

An edge case is an issue that occurs at an extreme (maximum or minimum) operating parameter.
A corner case is when multiple parameters are simultaneously at extreme levels, and the user is put at a corner of the configuration space.

And what about the term boundary case? My research suggests that it’s essentially interchangeable with edge case, which makes sense when you think about it…an edge defines a boundary between one thing and another, after all. Boundary case seems to more commonly find use in software engineering, where (again quoting from Wikipedia’s edge case entry):

In programming, an edge case typically involves input values that require special handling in an algorithm behind a computer program. As a measure for validating the behavior of computer programs in such cases, unit tests are usually created; they are testing boundary conditions of an algorithm, function or method. A series of edge cases around each “boundary” can be used to give reasonable coverage and confidence using the assumption that if it behaves correctly at the edges, it should behave everywhere else. For example, a function that divides two numbers might be tested using both very large and very small numbers. This assumes that if it works for both ends of the magnitude spectrum, it should work correctly in between.

Conversely, the edge case vernacular seemingly finds more common use with hardware engineers. Examples here, off the top of my tongue, include extremes in:

Operating temperature (including both ambient extremes and circuitry-generated heat, not to mention what happens when ventilation sources—i.e., fans and the like—fail)
Supply voltage and current
Electromagnetic interference, both self-created and ambient
Humidity, more blatant moisture exposure, and other environmental variables
Etc…

And for the mechanical engineers in the audience, a whole host of other variables beg for attention, involving pressure, torque and other measures of stress, vibration, and the like.

Let’s now revisit software. Putting aside obvious code bugs, such as accesses to invalid areas of system memory and the like, edge cases often involve input, intermediary and output data that’s other than expected. The information may be larger or smaller than what was comprehended by the coder; it might also be formatted differently than anticipated. And then there are cases such as one that I personally grappled with a few years ago…

The original version of the website for the Embedded Vision Alliance (now the Edge AI and Vision Alliance), my “day job” employer, was initially implemented in a now-archaic version of Drupal. As time went on, I’d increasingly grapple with content provided by a Member company (often written in Microsoft Word or another word processor, or originally published on their website in HTML) which, after I republished it, would (I kid you not) cause our website’s hosting server to spike CPU utilization, sometimes even locking up completely. The culprit, it turned out, was the source content’s inclusion of unconventional character sets and extended symbols as well as other characters within a set…specifically, emoji, which wasn’t in common use at the time of that Drupal version’s development and therefore hadn’t been comprehended by the coders.

Speaking of the Alliance…let’s focus now on software for systems that, paraphrasing the organization’s lingo: “perceive, understand and appropriately respond to their surroundings”. Semi- and fully-autonomous vehicles are one obvious example here, albeit a somewhat extreme one. Since they both contain human beings capable of being harmed or killed by a collision or other malfunction and are capable of colliding with other human beings (among other things), edge and corner case comprehension and testing should appropriately be far more extensive than, say, with an autonomous consumer drone that worst-case might collide with a tree or the side of a building, damaging nothing but itself in the process.

Just the other day I had a conversation with a colleague who relayed to me the story of an experience he’d just had; in traversing a shadow-filled underpass that also involved a “dip” in the roadway, his car had briefly but notably auto-braked, incorrectly perceiving an object ahead of it. More generally, for example, he said that the vehicle will refuse to back out of the garage if the driveway contains more than a few inches of snow, because it discerns the accumulation as something that it might adversely collide with. Due to this particular car’s age, I pointed out to him, its advanced driver assistance system (ADAS) algorithms were undoubtedly developed in a traditional manner, where the software engineer challengingly had to:

Brainstorm all possible edge/boundary and corner cases, and then
Explicitly code algorithms that comprehended and implemented correct responses

Nowadays, of course, autonomous vehicle (and more general autonomous mobile robot, or AMR) software development is deep learning-based, done quite differently. Instead of precisely coding algorithms to comprehend all possible usage scenarios, you instead “train” the deep learning model with an extensive data set increasingly including not only still and video images, but also “sensor fusion” data from radar, lidar, ultrasound, human hearing-range audio (other vehicles’ horns, for example), and the outputs of other sensing modalities.

As with traditional algorithm development, and reiterating my statement that opened this piece, whether or not (and if so how) to account for rarely encountered implementation variables remains an application-dependent balancing act (to my earlier contrast between vehicles and consumer drones). Feeding the model training function with an excess of images, for example, will invariably lead to an unnecessarily bloated model, not only consuming disproportionate system resources but also executing subsequent inference operations more slowly than would otherwise be the case…particularly an issue when rapid response is critical!

That said, it’s not just rarely encountered big-picture operational scenarios that require thoughtful inclusion consideration in upfront training, it’s also the data within each of these scenarios. A bouncing ball, or for that matter a distracted toddler in motion, looks completely different in the middle of the day than under more muted dawn or dusk lighting conditions (further complicated by rain, snow, fog, and other environmental attenuators). Not to mention at different distances from, and at broader varying orientations relative, to a viewing camera.

If you don’t have real-life images to cover all these scenarios—say, a human being directly facing a camera as well as oriented sideways, with his or her back to the camera lens and image sensor, in various sizes (both absolute and distance-determined), with various skin tones, and wearing various outfits—what do you do? Until recently, you relied on synthetic image generation to append the training data set, using tools not unlike those that video game developers harness. Not coincidentally, check out this presentation on synthetic data creation and training inclusion by Unity Technologies (a leading game engine developer) from the 2022 Embedded Vision Summit, a preview version of which I’ve embedded below:

Nowadays, however, generative AI is becoming increasingly powerful (as I noted in my 2023 forecast piece from last fall) and correspondingly is also becoming an increasingly tempting option for doing synthetic data generation. While sometimes the images it creates are fanciful:

other times its output is uncannily realistic:

And, of course, generative AI can find use not only in creating still images and video sequences but also sound clips and other data types. So why bother capturing (or at least collecting) a bunch of real-life data, or firing up your computer and tediously using audio, graphics and/or other tools to craft your own synthetic content for model training purposes? Why not instead just say “Market Street, San Francisco, CA, on a rare sunny day, including a trolley car” and have your preferred generative AI tool automatically synthesize exactly what you want?

The answer, it seems from recently published research, is that you should resist the temptation to do so, because it ends up being a really bad idea in how it affects the resultant quality of the trained model. As noted, for example, in VentureBeat’s coverage of the topic, aptly titled “The AI Feedback Loop: Researchers Warn of ‘Model Collapse’ as AI Trains on AI-generated Content”:

The data used to train the large language models (LLMs) and other transformer models underpinning products such as ChatGPT, Stable Diffusion and Midjourney comes initially from human sources — books, articles, photographs and so on — that were created without the help of artificial intelligence. Now, as more people use AI to produce and publish content, an obvious question arises: What happens as AI-generated content proliferates around the internet, and AI models begin to train on it, instead of on primarily human-generated content? A group of researchers from the UK and Canada have looked into this very problem and recently published a paper on their work in the open access journal arXiv. What they found is worrisome for current generative AI technology and its future: “We find that use of model-generated content in training causes irreversible defects in the resulting models.”

I’ll explain what I think may be going on by means of analogy to traditional computer vision. Images intended for human viewing purposes are often quite different than those optimized for computer vision analysis. In the former case, they’re intended to be perceived as pleasing to the human visual system, tailored for our green-dominant color perception scheme, for example, as well as to smooth out subjects’ skin blemishes, enhance detail in both dark and light areas of the image, etc. Conversely, images ideal for computer vision analysis have artificially enhanced edges (boundaries?), for example, that aid in differentiating one object in a scene from another…but at the same time can be perceived as undesirable to the human eye.

Analogously, what we perceive in a generative AI-synthesized “artificial” image and what a trained deep learning model might draw attention to might be very different. Minute variances between a real-life image of an automobile and a synthesized one might not be noticed by us—we might even prefer the artificial representation—but will only confuse a deep learning inference operation guided by the prior flawed model training process. And confusion leads to unintended results, including an increasingly documented phenomenon called hallucination:

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called confabulation or delusion) is a confident response by an AI that does not seem to be justified by its training data…Such phenomena are termed “hallucinations”, in loose analogy with the phenomenon of hallucination in human psychology. However, one key difference is that human hallucination is usually associated with false percepts, but an AI hallucination is associated with the category of unjustified responses or beliefs.

This writeup was intended to, and has hopefully succeeded in, providing you with plenty of “food for thought” as well as motivation for providing myself and your fellow readers with feedback. To wit, some questions for your consideration, to whet your appetite:

What examples from your past, present, and forecasted future product development experiences exist regarding corner, edge or whatever your favorite cases lingo is?
How do you know when to worry, or not, about accounting for a particular potential corner or edge case in your hardware and/or software design, what criteria guides that decision, and how does the outcome of your thought process vary over time, accumulated experience, situation specifics and other variables?
If you’re doing a deep learning-based implementation and you’re not confident that your existing model training data set is sufficiently comprehensive, how do you augment it? Conversely, if your training data set’s size and scope are overkill, how do you cull it?
Do you think that generative AI will end up being a boon, a bane, or some combination of the two in this regard?

I look forward to your thoughts in the comments!

—Brian Dipert is the Editor-in-Chief of the Edge AI and Vision Alliance, and a Senior Analyst at BDTI and Editor-in-Chief of InsideDSP, the company’s online newsletter.

VIDEO AD

–><!–

div-gpt-ad-inread

–>

<!–
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>

The post Edge and corner cases: AI’s conceptual simplifications and potential complications appeared first on EDN.

Edge and corner cases: AI’s conceptual simplifications and potential complications

Become a member

Become a subscriber

Become a sponsor