I worked for one of these firms starting in the late 80s, and they weren’t totally new then. There are different “types” of firms.
There are firms like McKinsey – they do a lot of strategic level consulting, helping with big picture issues and strategic analysis, complex modeling, things like that.
Then there are the big firms like Accenture, Deloitte, PWC, etc. – they have different divisions (usually a small strategy division), but an awful lot of the work they do is called integration work. When a company has a BIG project (say they are completely replacing a significant core software system or set of systems, or are getting into a new line of business. or trying to put together two companies in a large merger), they need extra resources – they usually have neither the management resources, the project management skills, or the bodies to do all the work to do stuff like make software configuration decisions and develop reports, and write all the needed conversion code and test all that stuff and develop new forms and write business procedures, and track & report progress on a huge project – whatever needs to be done. These firms provide a significant portion of the resources needed to get these big projects done, then (mostly) leave when the project is done. The hiring company can hire for varying components of the project (say, just the conversion for an acquisition, and they will do all needed software changes themselves while the consulting company is responsible for converting all the data). Or they can hire them for most of the project work, and split the way resources are provided (say consulting company provides 75% of resource, and hiring company provides 25%). It is uncommon for the consulting companies to take on small jobs (providing just a few resources), unless there is the potential to convince the client to hire for a bigger job.
So the employees of these companies typically move from project to project, and from client to client. Often they have little say in the location, industry, or role – a body is a body when they are trying to staff a project. But one of these projects can take years – although they are often in overlapping phases, and not all consultants stay for the whole project. Of course the more experiences consultants are more valuable, and the projects are seeded with those. They often do more of the client interaction. Newer people might be writing test cases or writing requirements or coding (although most of these firms now use offshore companies like Tata for at least some of the coding). As they gain experience, they might work on proposals for clients, project planning, and have more say up front in the decision making part of what software will be selected – things that require more experience and knowledge to do well.
Those companies do some other things – maybe take on big training projects (worldwide rollout of something new and large for an organization), sometimes they take over whole departments in a sort of outsourcing role, and they may have some niche specialties.
Then there are more niche consulting companies. Some of them work at a high level in a given industry – I have friends who work for a smallish highly regarded health care consulting company. They specialize in health care projects, and do a lot at the strategic level in that industry.
There are “body shops” that call themselves consulting firms (lots and lots and lots of those). They provide resources on a one-off basis – a couple project managers here, a business analyst there, a test coordinator, a program manager, a coder with specialized package or software language skills. But those firms are essentially only responsible for the resource. The other types of firms previously discussed often have some skin in the game for the completion and success of the project. The body shops are just that providing resources. They will replace them if they quit or prove incompetent or whatever. Maybe even not charge for all the hours in some situations like that. But they don’t tie billing to the project or business success.
And there are a lot of independent consultants. They usually form a relationship with a body shop, or sometimes one of the other types of firms, and work under their umbrella. The consulting company takes a cut of the hourly rate – deals vary depending on the industry, who found the contract, length of contract, etc.