The title says it all: Are halting and terminating synonyms in the context of computability theory?
Well, it's not called the Terminating Problem; it's called the "Halting Problem."
does not halt, but you could "terminate" it by kicking the power cord out of the wall (or clicking "End Task" in Task Manager), I would say, no, they are not synonyms.
More specifically relevant to halting,
So the difference between halting -- process does not continue indefinitely -- and terminating is that terminating implies returning a normal result. Not finishing abruptly (with an error or exceptional condition) or wedging itself in a state where no reduction rule applies.
This problem can be illustrated in the λ-calculus extended with booleans and numbers used by Benjamin Pierce:
which will not reduce because it gets stuck at
which does not match any reduction rule since the only rules that apply to