If we integrate both sides of:
$$\frac{d\epsilon}{dt} = \frac{d\sigma}{dt} \cdot \frac{1}{E} + \frac{\sigma}{\eta} \tag{1}$$
with respect to time, we get:
$$\int_0^t \frac{d\epsilon}{dt^\prime} dt^\prime= \int_0^t \frac{d\sigma}{dt^\prime} \cdot \frac{1}{E} dt^\prime+ \int_0^t \frac{\sigma}{\eta} dt^\prime\tag{2}.$$
It's fair to say that:
$$\int_0^t \frac{d\epsilon}{dt^\prime} dt^\prime= \epsilon\tag{3},$$
and if $E$ is constant with respect to time, it's fair to say that:
$$\int_0^t \frac{d\sigma}{dt^\prime} \cdot \frac{1}{E} dt^\prime= \frac{\sigma}{E}\tag{4}.$$
For the last term, if $\sigma$ and $\eta$ are constant with respect to time, then it's fair to say that:
$$\int_0^t \frac{\sigma}{\eta} dt^\prime = \frac{\sigma}{\eta}t\tag{5},$$
but if $\sigma$ is constant with respect to time, then the original equation might as well be:
$$\frac{d\epsilon}{dt} = \frac{\sigma}{\eta} \tag{6}$$
since we have:
$$\frac{d\sigma}{dt} = 0.\tag{7} $$
Based on Eq. 1, it's fair to assume that $\sigma$ depends on time, so Eqs 2-4 lead us to the following formula:
$$\epsilon(t)= \frac{\sigma(t)}{E} + \frac{1}{\eta}\int_0^t \sigma(t^\prime) dt^\prime\tag{8}.$$
If you know how $\sigma(t)$ depends on time, then you may be able to put Eq. 8 in "closed form", but if that were the case, Eq. 1 would likely be presented in a more simple way. The fact that Eq. 1 is written in the way that it's written, suggests that $\sigma$ has some time-dependence which prevents us from using simpler equations like the ones that you proposed.
