The "micro-laser" uses microscopic layer of material to produce photons of light, which are then amplified backwards and forwards between the layers into a laser beam.
This could help doctors minimise the amount of healthy tissue removed during operations - and make sure tumours have been completely cut out.
The "smart scalpel", developed by scientists at the US Department of Energy's Sandia National Laboratories, uses "nanotechnology" to check individual cells for signs of cancer.
The cells are pumped from near the cutting edge through a tiny laser.
Cancerous cells have a different chemical makeup to normal cells and change the laser light which passes through them.
This allows a computer to distinguish between the two.
Untested in real operations
So far the technology has only been tested successfully on cancer cells grown in the controlled conditions of the laboratory.
In a real operation, it would have to cope with a large amount of unwanted debris, along with cancer cells which appear very different to those grown in a petri dish.
Professor Garth Cruickshank, a professor of neurosurgery at Queen Elizabeth Hospital in Birmingham, said the idea of a "smart scalpel" was interesting, although he said he could not predict how much difference it would make to the survival of patients.
"Unless you get every single cancer cell removed - which you never do - other factors, such as the responsiveness of the tumour to chemotherapy or radiotherapy, will always be more important."
The cells involved were human brain cells and their malignant equivalents, called glioblastomas.
But those developing it believe it could help neurosurgeons accurately remove tumour tissue obscured by blood, muscle and fat.
Paul Gourley, who leads the project, said: "We can quickly identify a cell population that has abnormal protein content, as do tumour cells, by passing only a few hundred cells - a billionth of a litre - through our device.
"People didn't believe we could pump cells through a microlaser, make the cells part of the lasting process, and produce meaningful results.
"As it's turned out, we can do all these things."
The "micro-laser" uses microscopic layer of material to produce photons of light, which are then amplified backwards and forwards between the layers into a laser beam.
A micro-pump sends the cells through the beam one at a time - at a rate of 100,000 cells per second.
All this would take place in the tiny handle of the surgical scalpel, which would send the information back to a computer for analysis.
Other methods of analysing cells require more bulky equipment, are more expensive and take longer to return results.
The device is the result of two decades of work to create and use a laser on such a small scale.
It is important that the surgeon removes the whole tumour - otherwise it may grow back.
Similarly, if too much healthy tissue is cut away, particularly in the brain, this can have long-lasting effects.
The project team is currently in negotiations with biotechnology firms interested in marketing the scalpel.