Negative emotions can send us rushing to the biscuit tin or hiding our horrors in a tub of ice-cream - with up to 43% of people using food to alter their mood, according to a survey by the Priory Clinic.
Chocolate boosts serotonin levels, making us feel good... for a while
Others binge and vomit or develop anorexia as a way of trying to gain some sort of control over runaway feelings.
BBC News Online spoke to the experts to try to find out why so many of us have this up and down relationship with what, on its most fundamental level, is just a means of fuelling our bodies.
According to Deanne Jade, principle of the National Centre for Eating Disorders, our complex relationship with food starts right back in the cradle.
"A baby's blood sugar falls if they are hungry and they get in a dire emotional state. They scream and cry, are fed and feel better.
"So very early on we learn to associate stress reduction with eating and drinking.
"Secondly, the chemicals we gain from food have a strong impact not only on our appetite but also on our emotional brains, so the connections between food and mood become intricately connected in a very complex way."
Because of this complex relationship, people might say they are hungry and genuinely believe it, when in fact they are sad. They will also feel sad and miserable when they are hungry because of the deficiency of certain neurochemicals, which carry messages in the brain.
"For example, when we eat we get changes in serotonin, endorphins and dopamine levels in the brain. These are very powerful chemicals, which also affect mood.
"Sugar, for example, causes serotonin flows and serotonin is a chemical used in certain anti-depressants, such as Prozac, which cause the user to feel more balanced and equitable.
"People who are low in serotonin in the brain do generally feel better when they eat sugar, but that's not true of people who have adequate levels of serotonin."
Tempting - but ice cream is only a short-term answer to feeling bad
So could that mean we can become addicted to certain foods? The notion has certainly sparked vigorous debate in recent months.
Jane Ogden, a reader in health psychology at King's College, London, believes it depends on your definition of addiction.
She says: "If addiction means you have no control over something, then this could apply to food, but if you take the classic definition of addiction, which means you get withdrawal and tolerance, then I'm not sure it can really apply to eating behaviour."
She adds: "Ideally, if we were purely biological beings, we would only eat when we were hungry, but food intake isn't like that.
"Food becomes a central part of the currency of social life and also the individual's day-to-day emotional life.
"It is no longer is about whether you are hungry or not but takes on a range of other meanings. It becomes part of social negotiation and about celebration and special occasions, about relationships and power.
"The family meal has a whole context which has got very little to do with hunger and satiety and a lot more to do with communication within families and the way you show whether you care about people or how you are bonded.
"Then on an individual level, food has also taken on the role of comfort and distraction from boredom. If we were animals it would have only a very basic meaning but because we are social beings it is inevitable food is going to have a much bigger role in people's lives. "
Binge eating leads to a cycle that can prove hard to break
So how does all this apply to people with anorexia or bulimia?
"Starving causes people to feel high and spaced out and separated from their emotions," says Mrs Jade.
"When you starve, you don't feel the normal range of emotions. You feel kind of insulated from them. It doesn't mean you don't get depressed and miserable - you can get severely depressed - but we are talking about a narrowing of emotional range.
"There are many reasons why starving is alluring to some people. It makes them feel special, powerful, maybe more sexy, maybe less sexy, and it makes them feel they can transcend the normal things that other people need. It makes them feel very, very much in control.
"What is common to people with eating disorders is that body image and feelings about their body are a very big part of their self-esteem."
With binge eating, it has been proved that an unhealthy diet leads to chemical disturbances in the brain, setting up a cycle, so we become depressed, irritable or aggressive, then eat more of the very food that makes us feel that way.
Studies have shown that antisocial behaviour can be connected to having too much sugar and refined carbohydrates and a shortage of vitamins B11, B1 and minerals such as iron, selenium and magnesium and omega-3 fatty acids, found especially in oily fish such as salmon, sardines and mackerel.
A study published in the July 2002 issue of the British Journal of Psychiatry indicates that the simple use of nutritional supplements can significantly reduce behavioural problems among prisoners.
The research, carried out by a team led by C Bernard Gesch of Oxford University, was a trial of nutritional supplements on 231 young adults, comparing disciplinary offences before and during supplementation.
Those receiving the active capsules, rather than a placebo, committed an average of 26.3% fewer offences.
So in times of stress some people will eat more and some will eat less. It apparently comes down to something called biochemical individuality.
It also has a lot to do with the fact that we're all different. Our emotional response to food is formed in childhood, due to many things - the way our parents feed us, and our own experiences around food, which are very personal.
And as well as our chemistry and our individuality, the way we experience emotion and how we handle it also play a part.
To treat eating problems, the National Centre for Eating Disorders tries to regulate food intake, so people are at least protected on the chemical side. The centre also tries to help people become better at dealing with their own emotions without hiding behind food or drugs.
"If I got up in the morning, had a cup of black coffee, went to work and had a diet coke and had no protein until my evening meal, then washed it down with tea and biscuits, I'd be in a pretty bad emotional state by the end of the day as well," says Mrs Jade.
"And that is absolutely and totally to do with the food you don't eat, as well as the food you do eat. A lot of people are absolutely all over the place emotionally, simply because they have a desperately poor diet."