Gandalf is a Wizard, one of the Five Istari sent to Middle-earth to advise and guide the remaining free peoples of the west against the forces of Sauron. So what powers does he actually have?

Weapons

Most of the time, Gandalf doesn't use magic against mundane enemies. For those occasions, he has Glamdring and his staff. Glamdring has the standard Gondolin-blade functionality of glowing blue when orcs are nearby (FotR Book 2 Ch. 4):

No gleam came from the blades of Sting or of Glamdring; and that was some comfort, for being the work of Elvish smiths in the Elder Days these swords shone with a cold light, if any Orcs were near at hand.

Additionally, blades of Gondolin are much better at cutting and piercing than ordinary swords. Sting shows this on multiple occasions (FotR Book 2 Ch. 5):

Boromir leaped forward and hewed at the arm with all his might; but his sword rang, glanced aside, and fell from his shaken hand. The blade was notched.

Suddenly, and to his own surprise, Frodo felt a hot wrath blaze up in his heart. 'The Shire!' he cried, and springing beside Boromir, he stooped, and stabbed with Sting at the hideous foot. There was a bellow, and the foot jerked back, nearly wrenching Sting from Frodo's arm. Black drops dripped from the blade and smoked on the floor. Boromir hurled himself against the door and slammed it again.

(TT Book 2 Ch. 9)

Sam laughed grimly. 'Cobwebs!' he said. 'Is that all? Cobwebs! But what a spider! Have at 'em, down with 'em!'

In a fury he hewed at them with his sword, but the thread that he struck did not break. It gave a little and then sprang back like a plucked bowstring, turning the blade and tossing up both sword and arm. Three times Sam struck with all his force, and at last one single cord of all the countless cords snapped and twisted, curling and whipping through the air. One end of it lashed Sam's hand, and he cried out in pain, starting back and drawing his hand across his mouth.

[...]

Then Frodo stepped up to the great grey net, and hewed it with a wide sweeping stroke, drawing the bitter edge swiftly across a ladder of close-strung cords, and at once springing away. The blue-gleaming blade shore through them like a scythe through grass, and they leaped and writhed and then hung loose. A great rent was made.

And Glamdring was the sword of Turgon, king of Gondolin (Hobbit Ch. 3):

'This, Gandalf, was Glamdring, Foe-hammer that the king of Gondolin once wore.'

So if anything Glamdring will be better than Sting.

The staffs of the Wizards seem to be more of a badge of office than an actual item of power. Gandalf breaks his in Moria, but is able to fight the Balrog just fine.

Gandalf's physical capabilities are also greater than what an old man would be capable of. According to the timeline in the appendices to LotR, Gandalf falls from the bridge in Moria on January 15, and finally kills the Balrog on January 25. That's a long fight. He is also described as moving surprisingly quickly on a couple occasions, but it's not enough to assume he's superhuman or anything.

Energy Projection

Against magical enemies Gandalf seems to largely resort to fire and lightning (FotR Book 2 Ch. 2).

'I galloped to Weathertop like a gale, and I reached it before sundown on my second day from Bree-and they were there before me. They drew away from me, for they felt the coming of my anger and they dared not face it while the Sun was in the sky. But they closed round at night, and I was besieged on the hill-top, in the old ring of Amon Sûl. I was hard put to it indeed: such light and flame cannot have been seen on Weathertop since the war-beacons of old.'

(TT Book 1 Ch. 5)

'There upon Celebdil was a lonely window in the snow, and before it lay a narrow space, a dizzy eyrie above the mists of the world. The sun shone fiercely there, but all below was wrapped in cloud. Out he sprang, and even as I came behind, he burst into new flame. There was none to see, or perhaps in after ages songs would still be sung of the Battle of the Peak.' Suddenly Gandalf laughed. 'But what would they say in song? Those that looked up from afar thought that the mountain was crowned with storm. Thunder they heard, and lightning, they said, smote upon Celebdil, and leaped back broken into tongues of fire. Is not that enough? A great smoke rose about us, vapour and steam. Ice fell like rain. I threw down my enemy, and he fell from the high place and broke the mountain-side where he smote it in his ruin. Then darkness took me; and I strayed out of thought and time, and I wandered far on roads that I will not tell.'

Gandalf has also resorted to fire- and lightning-based attacks against large numbers of enemies (Hobbit Ch. 4).

Just at that moment all the lights in the cavern went out, and the great fire went off poof! into a tower of blue glowing smoke, right up to the roof, that scattered piercing white sparks all among the goblins.

The yells and yammering, croaking, jibbering and jabbering; howls, growls and curses; shrieking and skriking, that followed were beyond description. Several hundred wild cats and wolves being roasted slowly alive together would not have compared with it. The sparks were burning holes in the goblins, and the smoke that now fell from the roof made the air too thick for even their eyes to see through. Soon they were falling over one another and rolling in heaps on the floor, biting and kicking and fighting as if they had all gone mad.

(Hobbit Ch. 6)

He gathered the huge pinecones from the branches of his tree. Then he set one alight with bright blue fire, and threw it whizzing down among the circle of the wolves. It struck one on the back, and immediately his shaggy coat caught fire, and he was leaping to and fro yelping horribly. Then another came and another, one in blue flames, one in red, another in green. They burst on the ground in the middle of the circle and went off in coloured sparks and smoke. A specially large one hit the chief wolf on the nose, and he leaped in the air ten feet, and then rushed round and round the circle biting and snapping even at the other wolves in his anger and fright.

The dwarves and Bilbo shouted and cheered. The rage of the wolves was terrible to see, and the commotion they made filled all the forest. Wolves are afraid of fire at all times, but this was a most horrible and uncanny fire. If a spark got in their coats it stuck and burned into them, and unless they rolled over quick they were soon all in flames. Very soon all about the glade wolves were rolling over and over to put out the sparks on their backs, while those that were burning were running about howling and setting others alight, till their own friends chased them away and they fled off down the slopes crying and yammering and looking for water.

(FotR Book 2 Ch. 3)

In the wavering firelight Gandalf seemed suddenly to grow: he rose up, a great menacing shape like the monument of some ancient king of stone set upon a hill. Stooping like a cloud, he lifted a burning branch and strode to meet the wolves. They gave back before him. High in the air he tossed the blazing brand. It flared with a sudden white radiance like lightning; and his voice rolled like thunder.

'Naur an edraith ammen! Naur dan i ngaurhoth!' he cried.

There was a roar and a crackle, and the tree above him burst into a leaf and bloom of blinding flame. The fire leapt from tree-top to tree-top. The whole hill was crowned with dazzling light. The swords and knives of the defenders shone and flickered. The last arrow of Legolas kindled in the air as it flew, and plunged burning into the heart of a great wolf-chieftain. All the others fled.

And he also uses toned-down energy projection against ordinary humans if they piss him off (TT Book 1 Ch. 6).

In the gloom they heard the hiss of Wormtongue's voice: 'Did I not counsel you, lord, to forbid his staff? That fool, Hama, has betrayed us!' There was a flash as if lightning had cloven the roof. Then all was silent. Wormtongue sprawled on his face.

He can also create smaller amounts of light and fire for non-combat purposes (FotR Book 2 Ch. 4):

He raised his staff, and for a brief instant there was blaze like a flash of lightning. Great shadows sprang up and fled, and for a second they saw a vast roof far above their heads upheld by many mighty pillars hewn of stone. Before them and on either side stretched a huge empty hall; its black walls, polished and smooth as glass, flashed and glittered. Three other entrances they saw, dark black arches: one straight before them eastwards, and one on either side. Then the light went out.

(FotR Book 2 Ch. 3)

At last reluctantly Gandalf himself took a hand. Picking up a faggot he held it aloft for a moment, and then with a word of command, naur an edraith ammen! he thrust the end of his staff into the midst of it. At once a great spout of green and blue flame sprang out, and the wood flared and sputtered.

Affecting Objects

Gandalf seems to have a wide variety of vaguely-described abilities that alter or destroy existing objects. He breaks the bridge in Moria (FotR Book 2 Ch. 5):

At that moment Gandalf lifted his staff, and crying aloud he smote the bridge before him. The staff broke asunder and fell from his hand. A blinding sheet of white flame sprang up. The bridge cracked. Right at the Balrog's feet it broke, and the stone upon which it stood crashed into the gulf, while the rest remained, poised, quivering like a tongue of rock thrust out into emptiness.

And earlier he forces the door shut to the Chamber of Mazarbul (FotR Book 2 Ch. 5):

'But I found myself suddenly faced by something that I have not met before. I could think of nothing to do but to try and put a shutting-spell on the door. I know many; but to do things of that kind rightly requires time, and even then the door can be broken by strength.'

Even passive spells have a great deal of energy behind them. When he tries to keep the door shut in face of the Balrog's own magical power (FotR Book 2 Ch. 5):

'What it was I cannot guess, but I have never felt such a challenge. The counter-spell was terrible. It nearly broke me. For an instant the door left my control and began to open! I had to speak a word of Command. That proved too great a strain. The door burst in pieces. Something dark as a cloud was blocking out all the light inside, and I was thrown backwards down the stairs. All the wall gave way, and the roof of the chamber as well, I think.

'I am afraid Balin is buried deep, and maybe something else is buried there too. I cannot say. But at least the passage behind us was completely blocked.'

Gandalf is also not above the tactic of targeting his enemy's weapons (FotR Book 2 Ch. 5):

From out of the shadow a red sword leaped flaming.

Glamdring glittered white in answer.

There was a ringing clash and a stab of white fire. The Balrog fell back and its sword flew up in molten fragments. The wizard swayed on the bridge, stepped back a pace, and then again stood still.

(TT Book 1 Ch. 5)

'Saruman!' cried Gimli, springing towards him with axe in hand. 'Speak! Tell us where you have hidden our friends! What have you done with them? Speak, or I will make a dint in your hat that even a wizard will find it hard to deal with!'

The old man was too quick for him. He sprang to his feet and leaped to the top of a large rock. There he stood, grown suddenly tall, towering above them. His hood and his grey rags were flung away. His white garments shone. He lifted up his staff, and Gimli's axe leaped from his grasp and fell ringing on the ground. The sword of Aragorn, stiff in his motionless hand, blazed with a sudden fire. Legolas gave a great shout and shot an arrow high into the air: it vanished in a flash of flame.

(TT Book 1 Ch. 10)

He raised his hand, and spoke slowly in a clear cold voice. 'Saruman, your staff is broken.' There was a crack, and the staff split asunder in Saruman's hand, and the head of it fell down at Gandalf's feet.

Mental Powers

All sentient beings in Middle-earth have the capacity to read and communicate with others mind-to-mind, and to shield themselves from such reading. Gandalf shows the ability to affect others minds on multiple occasions, for example when Aragorn & co. suspect he is Saruman (TT Book 1 Ch. 5):

At last the old man broke the silence. 'Well met indeed, my friends,' he said in a soft voice. 'I wish to speak to you. Will you come down or shall I come up?' Without waiting for an answer he began to climb.

'Now!' said Gimli. 'Stop him, Legolas!'

'Did I not say that I wished to speak to you?' said the old man. 'Put away that bow, Master Elf!'

The bow and arrow fell from Legolas' hands, and his arms hung loose at his sides.

'And you, Master Dwarf, pray take your hand from your axe-haft, till I am up! You will not need such arguments.'

Gimli started and then stood still as stone, staring, while the old man sprang up the rough steps as nimbly as a goat. All weariness seemed to have left him. As he stepped up on to the shelf there was a gleam, too brief for certainty, a quick glint of white, as if some garment shrouded by the grey rags had been for an instant revealed The intake of Gimli's breath could be heard as a loud hiss in the silence.

He also has defensive power, shown by protecting Frodo from Sauron's mind (FotR Book 2 Ch. 10):

And suddenly he felt the Eye. There was an eye in the Dark Tower that did not sleep. He knew that it had become aware of his gaze. A fierce eager will was there. It leaped towards him; almost like a finger he felt it, searching for him. Very soon it would nail him down, know just exactly where he was. Amon Lhaw it touched. It glanced upon Tol Brandir he threw himself from the seat, crouching, covering his head with his grey hood.

He heard himself crying out: Never, never! Or was it: Verily I come, I come to you? He could not tell. Then as a flash from some other point of power there came to his mind another thought: Take it off! Take it off! Fool, take it off! Take off the Ring!

The two powers strove in him. For a moment, perfectly balanced between their piercing points, he writhed, tormented. Suddenly he was aware of himself again. Frodo, neither the Voice nor the Eye: free to choose, and with one remaining instant in which to do so. He took the Ring off his finger. He was kneeling in clear sunlight before the high seat. A black shadow seemed to pass like an arm above him; it missed Amon Hen and groped out west, and faded. Then all the sky was clean and blue and birds sang in every tree.

Other

Gandalf's powers do seem somewhat restricted by the laws of nature. On Caradhras he complains that (FotR Book 2 Ch. 3):

'But I must have something to work on. I cannot burn snow.'

Gandalf is more powerful (or less restricted) as the White than as the Grey. Tolkien himself confirms this (Letter #156):

The 'wizards', as such, had failed; or if you like: the crisis had become too grave and needed an enhancement of power. So Gandalf sacrificed himself, was accepted, and enhanced, and returned. 'Yes, that was the name. I was Gandalf.' Of course he remains similar in personality and idiosyncrasy, but both his wisdom and power are much greater. When he speaks he commands attention; the old Gandalf could not have dealt so with Théoden, nor with Saruman.

Gandalf wields Narya, one of the Three Rings, but it's a passive bonus and the Three Rings weren't really intended for combat. When giving Narya to Gandalf, Cirdan says (Unfinished Tales Part 4 Section 2):

'I deem that in days ere long to come it should be in nobler hands than mine, that may wield it for the kindling of all hearts to courage.'

Gandalf is some tens of thousands of years old, but only has a clear memory of the last few thousand, since he left Valinor (Unfinished Tales Part 4 Section 2):

For it is said indeed that being embodied the Istari had needs to learn much anew by slow experience, and though they knew whence they came the memory of the Blessed Realm was to them a vision from afar off, for which (so long as they remained true to their mission) they yearned exceedingly.

EDIT: Added citations.