In 1846, Fr Cavanagh was appointed to a Curacy in Westport, Co Mayo in the shadows of Croagh Patrick where he lived and worked for twenty-one years.  The beginning of his priestly ministry coincides with the Great Famine.  He at once addressed himself to the poverty and disease-ridden situation that confronted him.  His great love for the poor, his unbounded charity quickly manifested itself.  He even raffled his horse and watch to help the famine stricken. Frequently he had to anoint as many as forty dying parishioners before breakfast.

A remarkable story exists from his efforts to find a suitable building for the occupation of young girls. While a suitable building was erected the funds ran out and the work stopped.  One day it is related, when he was on the Lecanvey Road near Croagh Patrick, a stranger spoke to him about the new building and inquired the reason for its interruption.

Having been told by the young priest there was no money, the strange man handed him the exact amount necessary to complete it.  The donor’s name was never ascertained but the work was finished.  It was piously believed that the mysterious donor was none other than St Joseph.  The Sisters of Mercy of Westport gave this tribute to him in their convent records for 1867; “Our kind friend and holy guide the Rev B A Cavanagh was appointed to the parish of Knock.  This devoted priest was so good to the poor, so full of charity for all.  He had not only nothing of the world’s wealth but was greatly in debt”